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Abstract 


Information  flow  in  a  telecommunication  network  is  accomplished 
through  the  interaction  of  mechanisms  at  various  design  layers  with  the 
end  goal  of  supporting  the  information  exchange  needs  of  the  applica¬ 
tions.  In  wireless  networks  in  particular,  the  different  layers  interact  in 
a  nontrivial  manner  in  order  to  support  information  transfer.  In  this 
paper  we  will  present  abstract  models  that  capture  the  cross  layer  inter¬ 
action  from  the  physical  to  transport  layer  in  wireless  network  architec¬ 
tures  including  cellular,  ad-hoc  and  sensor  networks  as  well  as  hybrid 
wireless- wireline.  The  model  allows  for  arbitrary  network  topologies  as 
well  as  traffic  forwarding  modes,  including  datagrams  and  virtual  cir¬ 
cuits.  Furthermore  the  time  varying  nature  of  a  wireless  network,  due 
either  to  fading  channels  or  to  changing  connectivity  due  to  mobility,  is 
adequately  captured  in  our  model  to  allow  for  state  dependent  network 
control  policies.  Quantitative  performance  measures  that  capture  the 
quality  of  service  requirements  in  these  systems  depending  on  the  sup¬ 
ported  applications  are  discussed,  including  throughput  maximization, 
energy  consumption  minimization,  rate  utility  function  maximization 
as  well  as  general  performance  functionals.  Cross- layer  control  algo¬ 
rithms  with  optimal  or  suboptimal  performance  with  respect  to  the 
above  measures  are  presented  and  analyzed.  A  detailed  exposition  of 
the  related  analysis  and  design  techniques  is  provided. 


Contents 


1  Introduction  3 

2  The  Network  Model  and  Operational  Assumptions  8 

2.1  Link  Rate  Function  Examples  for  Different  Networks  10 

2.2  Routing  and  Network  Layer  Queueing  18 

2.3  Flow  Control  and  the  Transport  Layer  21 

2.4  Discussion  of  the  Assumptions  22 

3  Stability  and  Network  Capacity  25 

3.1  Queue  Stability  25 

3.2  The  Network  Layer  Capacity  Region  29 

3.3  The  Capacity  of  One  Hop  Networks  36 

4  Dynamic  Control  for  Network  Stability  40 

4.1  Scheduling  in  an  ON/OFF  Downlink  40 

4.2  Network  Model  44 


1 


2  Contents 


4.3  The  Stabilizing  Dynamic  Backpressure  Algorithm  47 

4.4  Lyapunov  Stability  50 

4.5  Lyapunov  Drift  for  Networks  55 

4.6  Time  Varying  Arrival  Rates  58 

4.7  Imperfect  Scheduling  58 

4.8  Distributed  Implementation  59 

4.9  Algorithm  Enhancements  and  Shortest  Path  Service  61 

4.10  Multi-commodity  Flows  and  Convex  Duality  64 

5  Networking  Outside  of  the  Capacity  Region: 

Utility  Optimization  and  Fairness  69 

5.1  The  Flow  Control  Model  and  Fairness  Objective  70 

5.2  Dynamic  Control  for  Infinite  Demand  73 

5.3  Performance  Analysis  81 

5.4  Flow  Control  for  Arbitrary  Input  Rates  91 

6  Networking  with  General  Costs  and  Rewards  100 

6.1  The  Network  Model  Assumptions  100 

6.2  Algorithm  Design  107 

6.3  Energy  Optimal  Networking  Examples  113 

6.4  A  Related  Algorithm  123 

7  Final  Remarks  129 

References  133 


1 


Introduction 


In  cross-layer  designs  of  wireless  networks,  a  number  of  physical  and  ac¬ 
cess  layer  parameters  are  jointly  controlled  and  in  synergy  with  higher 
layer  functions  like  transport  and  routing.  Furthermore  state  informa¬ 
tion  associated  with  a  specific  layer  becomes  available  across  layers  as 
certain  functions  might  benefit  from  that  information.  Typical  physical 
and  access  layer  functions  include  power  control  and  channel  alloca¬ 
tion,  where  the  latter  corresponds  to  carrier  and  frequency  selection  in 
OFDM,  spreading  code  and  rate  adjustment  in  spread  spectrum,  as  well 
as  time  slot  allocation  in  TDMA  systems.  Additional  choices  in  certain 
wireless  network  designs  may  include  the  selection  of  the  modulation 
constellation  or  the  coding  rate,  both  based  on  the  channel  quality  and 
the  desired  rates  [55]  [155].  Due  to  the  interference  properties  of  wire¬ 
less  communication,  the  communication  links  between  pairs  of  nodes 
in  a  multinode  wireless  environment  cannot  be  viewed  independently 
but  rather  as  interacting  entities  where  the  bit  rate  of  one  is  a  function 
of  choices  for  the  physical  and  access  layer  parameters  of  the  others. 
Our  cross-layer  model  in  this  paper  captures  the  interaction  of  these 
mechanisms,  where  all  the  physical  and  access  layer  parameters  are  col¬ 
lectively  represented  through  a  control  vector  /(f).  Another  intricacy  of 
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a  wireless  mobile  communication  network  is  the  fact  that  the  channel 
and  the  network  topology  might  be  changing  in  time  due  to  environ¬ 
mental  factors  and  user  mobility  respectively.  That  variation  might  be 
happening  at  various  time  scales  from  milliseconds  in  the  case  of  fast 
fading  to  several  seconds  for  connectivity  variations  when  two  nodes  get 
in  and  out  of  coverage  of  each  other  as  they  move.  Actions  at  different 
layers  need  to  be  taken  depending  on  the  nature  of  the  variability  in 
order  for  the  network  to  compensate  in  an  optimal  manner.  All  the 
relevant  parameters  of  the  environment  that  affect  the  communication 
are  represented  in  our  model  by  the  topology  state  variable  5(f). The 
topology  state  might  not  be  fully  available  to  the  access  controller, 
who  may  observe  only  a  sufficient  statistic  of  that.  The  collection  of  bit 
rates  of  all  communicating  pairs  of  nodes  at  each  time,  i.e.  the  com¬ 
munication  topology,  is  represented  by  a  function  C(t)  =  C(I(t ),  5(f)). 
Note  that  the  function  C(.,.)  incorporates  among  others  the  depen¬ 
dence  of  the  link  rate  on  the  Signal-to-Noise  plus  Interference  Ratio 
(SNIR)  through  the  capacity  function  of  the  link.  Over  the  virtual  com¬ 
munication  topology  defined  by  C'(f),  the  traffic  flows  from  the  origin 
to  the  destination  according  to  the  network  and  transport  layer  pro¬ 
tocols.  Packets  may  be  generated  at  any  network  node  having  as  final 
destination  any  other  network  node,  potentially  several  hops  away.  Fur¬ 
thermore  the  traffic  forwarding  might  be  either  datagram  or  based  on 
virtual  circuits  while  multicast  traffic  may  be  incorporated  as  well.  The 
above  model  captures  characteristics  and  slightly  generalizes  systems 
that  have  been  proposed  and  studied  in  several  papers  including  [142] 
[147]  [148]  [144]  [135]  [108]  [122]  [119]  [136].  That  model  is  developed 
in  detail  in  Chapter  2  while  representative  examples  of  typical  wireless 
models  and  architectures  that  fit  within  its  scope  are  discussed  there. 

The  network  control  mechanism  determines  the  access  control  vec¬ 
tor  and  the  traffic  forwarding  decisions  in  order  to  accomplish  certain 
objectives.  The  quantitative  performance  objectives  should  reflect  the 
requirements  posed  by  the  applications.  Various  objectives  have  been 
considered  and  studied  in  various  papers  including  the  overall  through¬ 
put,  power  optimization,  utility  optimization  of  the  allocated  rates  as 
well  as  optimization  of  general  objective  functions  of  throughput  and/or 
power.  In  the  current  paper  we  present  control  strategies  for  achieving 
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these  objectives. 

The  first  performance  attribute  considered  is  the  capacity  region  of 
the  network  defined  as  the  set  of  all  end-to-end  traffic  load  matrices 
that  can  be  supported  under  the  appropriate  selection  of  the  network 
control  policy.  That  region  is  characterized  in  two  stages.  First  the 
ensemble  of  all  feasible  long  term  average  communication  topologies 
is  characterized.  The  capacity  region  includes  all  traffic  load  matrices 
such  that  there  is  a  communication  topology  from  the  ensemble  for 
which  there  is  a  flow  that  can  carry  the  traffic  load  and  be  feasible  for 
the  particular  communication  topology.  Chapter  3  is  devoted  on  the 
characterization  of  the  capacity  region  outlined  above. 

The  capacity  region  of  the  network  should  be  distinguished  from 
the  capacity  region  of  a  specific  policy.  The  latter  being  the  collection 
of  all  traffic  load  matrices  that  are  sustainable  by  the  specific  policy. 
Clearly  the  capacity  region  of  the  network  is  the  union  of  the  individ¬ 
ual  policy  capacity  regions,  taken  over  all  possible  control  policies.  One 
way  to  characterize  the  performance  of  a  policy  is  by  its  capacity  region 
itself.  The  larger  the  capacity  region  is  the  better  the  performance  will 
be  since  the  network  will  be  stable  for  a  wider  range  of  traffic  loads 
and  therefore  more  robust  to  traffic  fluctuations.  Such  a  performance 
criterion  makes  even  more  sense  in  the  context  of  wireless  ad-hoc  net¬ 
works  where  both  the  traffic  load  as  well  as  the  network  capacity  may 
vary  unpredict  ably.  A  policy  A  is  termed  "better"  than  B  with  respect 
to  their  capacity  regions,  if  the  capacity  region  of  A  is  a  superset  of 
the  capacity  region  of  B.  A  control  policy  that  is  optimal  in  the  sense 
of  having  a  capacity  region  that  coincides  with  the  network  capacity 
region  and  is  therefore  a  superset  of  the  capacity  region  of  any  other 
policy  was  introduced  in  [142]  [147].  That  policy,  the  max  weight  adap¬ 
tive  back-pressure  policy,  was  generalized  later  in  several  ways  [144] 
[135]  [108]  [122]while  it  is  essential  component  of  policies  that  optimize 
other  performance  objectives.  It  is  presented  in  Chapter  4.  The  selec¬ 
tion  of  the  various  control  parameters,  from  the  physical  to  transport 
layer  is  done  in  two  stages  in  the  max  weight  adaptive  back  pressure 
policy.  In  the  first  stage  all  the  parameters  that  affect  the  transmission 
rates  of  the  wireless  links  are  selected,  i.e.  the  function  C(I(t),  S(t))  is 
determined.  In  the  second  stage  routing  and  flow  control  decisions  to 
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control  multihop  traffic  forwarding  are  made.  The  back  pressure  policy 
consist  in  giving  priority  in  forwarding  through  a  link  to  traffic  classes 
that  have  higher  backlog  differentials.  Furthermore  the  transmission 
rate  of  a  link  that  leads  to  highly  congested  regions  of  the  network  is 
throttled  down.  In  that  manner  the  congestion  notification  travel  back¬ 
wards  all  the  way  to  the  source  and  flow  control  is  performed.  Proofs 
of  the  results  based  on  Lyapunov  stability  analysis  are  presented  also 
in  Chapter  4. 

The  stochastic  optimal  control  problem  where  the  objective  is  the 
optimization  of  a  performance  functional  of  the  system  is  considered 
in  Chapters  5,  6.  The  development  of  optimal  policies  for  these  cases 
relies  on  a  number  of  advances  including  extensions  of  Lyapunov  tech¬ 
niques  to  enable  simultaneous  treatment  of  stability  and  performance 
optimization,  introduction  of  virtual  cost  queues  to  transform  perfor¬ 
mance  constraints  into  queueing  stability  problems  and  introduction  of 
performance  state  queues  to  facilitate  optimization  of  time  averages. 
These  techniques  have  been  developed  in  [108]  [109]  [119]  [136]  [137] 
[45]  for  various  performance  objectives.  More  specifically  in  Chapter  5 
the  problem  of  optimizing  a  sum  of  utility  functions  of  the  rates  al¬ 
located  to  the  different  traffic  flows  is  considered.  That  formulation 
includes  the  case  of  the  traffic  load  in  the  system  being  out  of  the  ca¬ 
pacity  region  in  which  case  some  kind  of  flow  control  at  the  edges  of  the 
network  needs  to  be  employed.  That  is  done  implicitly  through  the  use 
of  performance  state  queues,  allowing  to  adjust  the  optimization  accu¬ 
racy  through  a  parameter.  The  approach  combines  techniques  similar 
to  those  used  for  optimization  of  rate  utility  functions  in  window  flow 
controlled  sessions  in  wireline  networks,  with  max  weight  scheduling 
for  dealing  with  the  wireless  scheduling.  In  Chapter  6  generalization 
of  these  techniques  for  optimization  functionals  that  combine  utilities 
with  other  objectives  like  energy  expenditure  are  given  and  approaches 
relying  on  virtual  cost  queues  are  developed. 

Most  of  the  results  presented  in  the  paper  are  robust  on  the  statis¬ 
tics  of  the  temporal  model  both  of  the  arrivals  as  well  as  the  topology 
variation  process.  The  traffic  generation  processes  might  be  Markov 
modulated  or  belong  to  a  sample  path  ensemble  that  comply  to  cer¬ 
tain  burstiness  constraints  [36]  [143].  Similarly  the  variability  of  the 
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topology  might  be  modeled  by  a  hidden  Markov  process.  These  models 
are  adequate  to  cover  most  of  the  interesting  cases  that  might  arise  in 
real  cases.  The  proofs  in  the  paper  are  provided  for  a  traffic  generation 
model  that  covers  all  the  above  cases  and  it  was  considered  in  [108]. 
The  definition  of  stability  that  was  used  implies  bounded  average  back¬ 
logs.  The  emphasis  in  the  presentation  is  on  describing  the  models  and 
the  algorithms  with  application  examples  that  illustrate  the  range  of 
possible  applications.  Representative  cases  are  analyzed  in  full  detail 
to  illustrate  the  applicability  of  the  analysis  techniques,  while  in  other 
cases  the  results  are  described  without  proofs  and  references  to  the 
literature  are  provided. 


The  Network  Model  and  Operational  Assumptions 


Consider  a  general  network  with  a  set  M  of  nodes  and  a  set  £  of  trans¬ 
mission  links.  We  denote  by  N  and  L  respectively  the  number  of  nodes 
and  links  in  the  network.  Each  link  represents  a  communication  channel 
for  direct  transmission  from  a  given  node  a  to  another  node  b,  and  is 
labeled  by  its  corresponding  ordered  node  pair  (a,  b )  (where  a,  b  6  A f). 
Note  that  link  (a,  6)  is  distinct  from  link  (6,  a).  In  a  wireless  network, 
direct  transmission  between  two  nodes  may  or  may  not  be  possible 
and  this  capability,  as  well  as  the  transmission  rate,  may  change  over 
time  due  to  weather  conditions,  mobility  or  node  interference.  Hence 
in  the  most  general  case  one  can  consider  that  £  consists  of  all  ordered 
pairs  of  nodes,  where  the  transmission  rate  of  link  (a,  b)  is  zero  if  direct 
communication  is  impossible.  However,  in  cases  where  direct  commu¬ 
nication  between  some  nodes  is  never  possible,  it  is  helpful  to  consider 
that  £  is  a  strict  subset  of  the  set  of  all  ordered  pairs  of  nodes. 

The  network  is  assumed  to  operate  in  slotted  time  with  slots 
normalized  to  integral  units,  so  that  slot  boundaries  occur  at  times 
t  €  {0, 1,2,.. .}.  Hence,  slot  t  refers  to  the  time  interval  [t,  t  +  1).  Let 
n(t)  =  (Hab(t))  represent  the  matrix  of  transmission  rates  offered  over 
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each  link  (a,  b)  during  slot  t  (in  units  of  bits/slot).1  By  convention,  we 
define  fiab{t)  =  0  for  all  time  t  whenever  a  physical  link  (a,  b )  does  not 
exist  in  the  network.  The  link  transmission  rates  are  determined  by  a 
link  transmission  rate  function  C(I,S),  so  that: 

where  S(t)  represents  the  network  topology  state  during  slot  t,  and  I(t) 
represents  a  link  control  action  taken  by  the  network  during  slot  t. 

The  topology  state  process  S(t)  represents  all  uncontrollable  prop¬ 
erties  of  the  network  that  influence  the  set  of  feasible  transmission 
rates.  For  example,  the  network  channel  conditions  and  interference 
properties  might  change  from  time  to  time  due  to  user  mobility,  wire¬ 
less  fading,  changing  weather  locations,  or  other  external  environmen¬ 
tal  factors.  In  such  cases,  the  topology  state  S(t)  might  represent  the 
current  set  of  node  locations  and  the  current  attenuation  coefficients 
between  each  node  pair.  While  this  topology  state  S(t)  can  contain  a 
large  amount  of  information,  for  simplicity  of  the  mathematical  model 
we  assume  that  S(t )  takes  values  in  a  finite  (but  arbitrarily  large)  state 
space  S.  We  assume  that  the  network  topology  state  S(t)  is  constant  for 
the  duration  of  a  timeslot,  but  potentially  changes  on  slot  boundaries. 

The  link  control  input  I(t)  takes  values  in  a  general  state  space  %s(t)> 
which  represents  all  of  the  possible  resource  allocation  options  available 
under  a  given  topology  state  S(t).  For  example,  in  a  wireless  network 
where  certain  groups  of  links  cannot  be  activated  simultaneously,  the 
control  input  I(t)  might  specify  the  particular  set  of  links  chosen  for 
activation  during  slot  t.  and  the  set  I sit)  could  represent  the  collection 
of  all  feasible  link  activation  sets  under  topology  state  S(t ).  In  a  power 
constrained  network,  the  control  input  I(t)  might  represent  the  matrix 
of  power  values  allocated  for  transmission  over  each  data  link.  Likewise, 
the  transmission  control  input  I(t)  might  include  bandwidth  allocation 
decisions  for  every  data  link. 

Every  timeslot  the  network  controller  observes  the  current  topol- 

1  Transmission  rates  can  take  units  other  than  bits/slot  whenever  appropriate.  For  example, 
in  cases  when  all  data  arrives  as  fixed  length  packets  and  transmission  rates  are  constrained 
to  integral  multiples  of  the  packet  size,  then  it  is  often  simpler  to  let  /x(t)  takes  units  of 
packet  s/slot. 
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ogy  state  S(t)  and  chooses  a  transmission  control  input  1(f)  6  XS{t), 
according  to  some  transmission  control  policy.  This  enables  a  trans¬ 
mission  rate  matrix  of  fx(t)  =  C(I(t),  S(t)).  The  function  C(I,S)  is 
matrix  valued  and  is  composed  of  individual  Cab(I,S)  functions  that 
specify  the  individual  transmission  rates  on  each  link  (a,  b),  so  that 
Habit)  =  Cab(I(t),S(t)).  In  general,  the  rate  function  for  a  single  link 
can  depend  on  the  full  control  input  /(f)  and  the  full  topology  state 
S(t)  and  hence  distributed  implementation  may  be  difficult.  This  is  of¬ 
ten  facilitated  when  rate  functions  for  individual  links  depend  only  on 
the  local  control  actions  and  the  local  topology  state  information  asso¬ 
ciated  with  those  links.  These  issues  will  be  discussed  in  more  detail  in 
later  sections. 

2.1  Link  Rate  Function  Examples  for  Different  Networks 

In  this  section  we  consider  different  types  of  networks  and  their  corre¬ 
sponding  link  rate  functions  C(/(t),  S(t)).  Our  examples  include  static 
wireline  networks,  rate  adaptive  wireless  networks,  and  ad-hoc  mobile 
networks. 


Example  2.1.  A  static  wireline  network  with  fixed  link  capacities. 
Consider  the  six  node  network  of  Fig.  2.1a.  The  network  is  connected 
via  wired  data  links,  where  each  link  (a,  b )  offers  a  fixed  transmission 
rate  Cab  for  all  time.  In  this  case,  there  is  no  notion  of  a  time  varying 
topology  state  S(t)  or  a  control  input  /(f),  and  so  the  transmission 
rate  function  for  each  link  (a,  b)  is  given  by  Cab{I(t),  S{t))  =  Cab 
(where  Cab  =  0  if  there  is  no  link  from  node  a  to  node  b).  The 
network  is  thus  fully  described  by  a  constant  matrix  (Cab)  of  link 
capacities,  which  is  the  conventional  way  to  describe  a  wireline  network. 


Example  2.2.  A  network  with  time  varying  link  capacities.  Consider 
the  same  network  as  in  Example  2.1,  but  assume  now  that  every  times¬ 
lot  the  data  links  can  randomly  become  active  or  inactive.  In  particular, 
an  active  link  (a,  b )  can  transmit  at  rate  Cab  as  before,  but  an  inactive 
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Fig.  2.1  (a)  A  static  network  with  6  nodes  and  constant  link  capacities  Cab .  (b)  A  network 
with  configurable  link  activation  sets. 


link  cannot  transmit.  Let  Sab(t)  be  a  link  state  process  taking  values 
in  the  two-element  set  {ON,  OFF},  where  Sab(t )  =  ON  if  link  (a,  b)  is 
active  during  slot  t,  and  Sab(t )  =  OFF  otherwise.  The  topology  state 
S(t )  of  the  network  is  thus  the  matrix  ( Sab(t ))  composed  of  individual 
link  state  processes,  and  the  link  transmission  rate  functions  are  given 
by  Cab(I{t),S(t))  =  Cab(Sab{.t )),  where: 


Cab(Sab(t)) 


Cab  if  Sab(t)  =  ON 
0  if  Sab(t)  =  OFF 


In  this  example,  the  link  transmission  rate  function  depends  on  a 
time  varying  topology  state  variable  S(t),  but  there  is  still  no  notion 
of  resource  allocation.  Further  note  that  the  stochastics  of  the  link 
activation  processes  Sab(t)  are  not  specified  here.  A  simple  model 
might  be  that  each  process  Sab(t )  is  independently  inactive  with  some 
outage  probability  pab  every  slot,  and  active  otherwise.  However,  in 
general,  the  Sab(t )  processes  could  be  correlated  with  each  other  and 
also  correlated  in  time. 


Example  2.3.  A  static  wireless  network  with  configurable  link  activa¬ 
tion  sets.  Consider  a  wireless  network  with  stationary  nodes  and  time 
invariant  channel  conditions  between  each  node  pair.  Suppose  that  due 
to  interference  and/or  hardware  constraints,  transmission  over  a  link 
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can  take  place  only  if  certain  constraints  are  imposed  on  transmissions 
over  the  other  links  in  the  network.  For  example,  a  node  may  not  trans¬ 
mit  and  receive  at  the  same  time  over  some  of  its  attached  links,  or  a 
node  may  not  transmit  when  a  neighboring  node  is  receiving,  etc.  A 
given  link  (a,  b)  can  support  a  transmission  rate  Cab,  provided  that  it  is 
scheduled  for  activation  and  no  other  interfering  links  are  activated.  For 
each  link  (a,  6),  we  define  a  control  process  Iab(t),  where  Iab(t)  =  1  if 
link  (a,  b)  is  activated  during  slot  t,  and  0  else.  The  control  input  process 
I(t )  thus  consists  of  the  matrix  ( Iab(t )),  and  this  matrix  is  restricted 
every  timeslot  to  the  set  X  consisting  of  all  feasible  link  activation  sets. 
That  is,  the  set  X  contains  all  sets  of  links  that  can  be  simultaneously 
activated  without  creating  inter-link  interference.  The  link  transmission 
rate  function  is  thus  given  by  Cab(I(t),  S(t))  =  Cab(Iab{t )),  where: 

(t  /  Cab  if  Iab(t)  =  1 

CabVabit))  =  |  Q  ,f  Iab{t)  =  Q  . 

where  the  input  satisfies  the  constraint  ( Iab(t ))  G  X  for  all  t.  An  ex¬ 
ample  network  with  three  activated  links  is  shown  in  Fig.  2.1b.  While 
this  link  transmission  rate  function  is  similar  in  structure  to  that  of 
Example  2.2,  we  note  that  the  link  capacities  of  Example  2.2  depend 
on  random  and  uncontrollable  channel  processes,  while  the  link  capac¬ 
ities  in  this  example  are  determined  by  the  network  control  decisions 
made  every  timeslot.  This  is  an  important  distinction,  and  the  notion 
of  link  activation  sets  can  be  used  to  model  general  problems  involving 
network  server  scheduling.  Such  problems  are  treated  in  [147]  for  multi¬ 
hop  radio  networks  with  general  activation  sets  X.  An  interesting  special 
case  is  when  X  is  defined  as  the  collection  of  all  link  sets  such  that  no 
node  is  the  transmitter  or  receiver  of  more  than  one  link  in  the  set. 
Such  sets  of  links  are  called  matchings.  This  special  case  has  been  used 
extensively  in  the  literature  on  crossbar  constrained  packet  switches, 
where  the  network  nodes  are  arranged  according  to  a  bipartite  graph 
(see  for  example,  [147]  [104]  [86]  [145]  [162]  [120]  [118]).  Matchings  are 
also  used  in  [145]  [61]  [92]  [29]  [163]  to  treat  scheduling  in  computer  sys¬ 
tems  and  ad-hoc  networks  with  arbitrary  graph  structures.  Note  that 
there  is  an  inherent  difficulty  in  implementing  control  decisions  in  a 
distributed  manner  under  this  model.  Indeed,  the  constraint  7(f)  G  X 
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couples  the  link  activation  decisions  at  every  node,  and  often  exten¬ 
sive  message  passing  is  required  before  a  matching  is  computed  and  its 
feasibility  is  verified.  Generally,  the  complexity  associated  with  finding 
a  valid  matching  increases  with  the  size  of  the  network.  Complexity 
can  also  be  reduced  by  considering  sub-optimal  matchings,  which  often 
yields  throughput  within  a  certain  factor  of  optimality.  This  approach 
is  considered  in  [92]  [163]  [29]  (see  also  Section  4.7). 


Example  2.4.  A  time  varying  wireless  downlink.  Consider  a  single 
wireless  node  that  transmits  to  M  downlink  users  (such  as  a  satellite 
unit  or  a  wireless  base  station,  see  Fig.  2.2a).  Let  Si(t)  represent  the 
condition  of  downlink  i  during  slot  t  (for  each  link  i  G  {1 , ,M}).  Sup¬ 
pose  that  channel  conditions  are  grouped  into  four  categories,  so  that: 
Si(t)  G  {GOOD,  MEDIUM,  BAD,  ZERO}.  Suppose  that  at  most 
one  link  can  be  activated  during  any  slot,  and  that  an  active  link  can 
transmit  3  packets  when  in  the  GOOD  state,  2  packets  in  the  MEDIUM 
state,  1  in  the  BAD  state,  and  none  in  the  ZERO  state.  The  topology 
state  S(t)  for  this  system  is  given  by  the  vector  (S'i(t), . . . ,  5m  (i))-  The 
control  input  I(t)  is  given  by  the  vector  . . . ,  /m(0),  where  I;(i) 

takes  the  value  1  if  link  i  is  activated  in  slot  t,  and  zero  else.  The  con¬ 
trol  space  2  is  the  set  of  all  vectors  {I\, . . . ,  Im)  with  at  most  one  entry 
equal  to  1  and  all  other  entries  equal  to  zero.  As  there  is  only  a  single 
transmitting  node,  we  can  express  the  link  transmission  rate  function 

as  a  vector:  C(I(t),S(t))  =  (Ci(I(t),  S(t)), _ ,  Cm(I(£),  S(t))).  Each 

function  entry  has  the  form  Ci(I(t),  S(t))  =  Ci(Ii(t),  Si(t)),  where: 


CiVi^Si®) 


'  31;  (t)  if  Si(t)  =  GOOD 
21; (f )  if  Si{t)  =  MEDIUM 
II; \t)  if  Si{t)  =  BAD 
0  else 


This  type  of  downlink  model  is  used  to  treat  satellite  and  wireless  sys¬ 
tems  in  [148]  [96]  [5]  [121].  The  model  can  be  extended  to  include  power 
allocation  in  cases  when  transmission  rates  depend  upon  a  continuous 
power  parameter  [121].  For  example,  the  transmission  rate  on  each 
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Fig.  2.2  (a)  An  example  satellite  downlink  with  M  downlink  channels  (M  =  7  in  the 
example),  (b)  An  example  set  of  rate-power  curves  for  the  power  allocation  problem  with 
four  discrete  channel  states. 


downlink  i  €  {1, . . . ,  M}  might  be  approximated  by  the  expression  for 
Shannon  capacity  over  an  additive  white  Gaussian  noise  channel: 

Ci(Pi(t),  Si(t ))  =  log(l  +  Pi(t)am),  (2.1) 

where  Pi(t)  is  the  power  allocated  to  channel  i  during  timeslot  t.  and 
c *Si(t )  is  the  attenuation-to- noise  coefficient  associated  with  channel 
state  Si(t)  (see  Fig.  2.2b).  In  this  case,  the  control  input  I(t)  is  given 
by  the  power  vector  P(t)  =  (Pi(t), . . . ,  Pm(£))>  and  the  control  space 
can  be  a  continuum  of  feasible  power  vectors,  such  as  all  vectors  that 
satisfy  a  peak  power  constraint  X)*=i  Pi  —  Ppeak- 


Example  2.5.  A  time  varying  ad-hoc  network  with  interference.  Con¬ 
sider  an  ad-hoc  wireless  network  with  a  set  of  nodes  A f  and  set  of  links 
C.  We  assume  that  each  link  l  =  (a,  b),  has  a  transmitter  located  at 
node  a  and  a  receiver  located  at  node  b.  Let  Pi(t)  represent  the  power 
that  the  transmitter  of  link  l  allocates  for  transmission  over  that  link, 
and  let  P(t)  =  ( P/(d)ze£  represent  the  power  allocation  vector.  In  this 
case,  the  control  input  I(t)  is  equal  to  the  power  vector  P(t).  and  the 
constraint  set  I  is  given  by  the  set  V  consisting  of  all  power  vectors 
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that  satisfy  peak  power  constraints  at  every  node.  The  transmission 
rate  function  for  link  l  is  given  by  Ci(I(t),  S(t))  =  Ci(P(t),  S(t)).  As¬ 
sume  that  this  function  depends  on  the  overall  Signal  to  Interference 
plus  Noise  Ratio  (SINR)  according  to  a  logarithmic  capacity  curve: 

Ci(P(t ),  S(t ))  =  log  (1  +  SINR^Pit),  5(t))) . 


Here  SINRi(P(t),S(t))  is  given  by: 


SINR^P^^it)) 


Pi(t)au(S{t )) 

No  +  Pk(t)aki(S(t ))  ’ 

k^l 


where  Nq  is  the  background  noise  intensity  on  each  link  and  aki{S(t)) 
is  the  attenuation  factor  at  the  receiver  of  link  l  of  the  signal  power 
transmitted  by  the  transmitter  of  link  k  when  the  topology  state  is 
S(t).  Hence,  in  this  model  the  interference  caused  at  the  receiver  of 
link  l  by  the  signals  transmitted  by  the  transmitters  of  the  other  links 
in  modeled  as  additional  noise.  This  SINR  network  model  is  quite 
common  in  the  wireless  and  ad-hoc  networking  literature.  For  example, 
[122]  considers  this  model  for  mobile  ad-hoc  networks,  and  [43]  [66] 
[35]  [93]  [127]  [128]  [129]  [31]  [167]  [123]  [124]  [171]  for  static  ad-hoc 
networks  and  cellular  systems.  This  model  in  the  case  of  a  system  with 
antenna  arrays  and  beamforming  capabilities  is  considered  in  [48]  [47] 
[28]  [130].  It  is  quite  challenging  to  implement  optimal  controllers  for 
this  type  of  link  transmission  rate  function.  Indeed,  as  in  Example 
2.3,  the  control  input  decisions  are  coupled  at  every  node,  because 
the  power  allocated  for  a  particular  data  link  can  act  as  interference 
at  all  other  links,  and  this  interference  model  can  change  depending 
on  the  network  topology  state.  While  distributed  algorithms  exist  for 
computing  the  rate  associated  with  a  particular  power  allocation,  and 
for  determining  if  a  power  allocation  exists  that  leads  to  a  given  set 
of  link  rates  [167]  [171],  there  are  no  known  low  complexity  algorithms 
for  finding  the  power  vectors  that  optimize  the  performance  metrics 
required  for  optimal  network  control.  However,  randomized  distributed 
approximations  exist  for  such  systems  and  offer  provable  performance 
guarantees  [57]  [108]  [122],  Furthermore,  important  special  cases  of  the 
low  SINR  regime  are  treated  in  [35]  [127]  [129]  using  the  approximation 
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log(l  +  SINR )  «  SINR ,  and  the  high  SINR  regime  is  treated  in  [66] 
[31]  using  the  approximation  log(l  +  SINR )  ps  log(SINR). 


Example  2.6.  An  ad-hoc  mobile  network.  Consider  a  network  with  a 
set  N  of  mobile  users.  The  location  of  each  user  is  quantized  accord¬ 
ing  to  a  rectilinear  cell  partitioning  that  covers  the  network  region  of 
interest,  as  shown  in  Fig.  2.3b.  We  assume  that  the  channel  conditions 
(noise,  attenuation  factor)  are  time-invariant  throughout  the  region  so 
that  link  transmission  capabilities  are  determined  solely  by  node  loca¬ 
tions.  Let  Sa(t)  represent  the  cell  location  of  node  a  during  slot  t.  The 
topology  state  variable  S(t)  consists  of  the  vector  (S'a(t))agA/-  (one  com¬ 
ponent  for  each  node  a  £  A f) ,  and  can  change  from  slot  to  slot  as  nodes 
move  from  cell  to  cell  (according  to  some  mobility  process  that  is  poten¬ 
tially  different  for  every  node).  In  this  case,  the  link  transmission  rate 
function  can  be  given  by  the  SINR  model  of  Example  2.5,  where  the 
attenuation  coefficients  otki(S(t))  are  determined  by  the  current  node 
locations.  Note  that  the  mobility  model  has  been  left  unspecified.  Any 
desired  mobility  model  can  be  used,  such  as  Markovian  random  walks 
[122],  periodic  walks,  random  waypoint  mobility  [25],  independent  cell 
hopping  [117]  [91],  etc.  The  network  model  can  be  simplified  by  assum¬ 
ing  no  inter- cell  interference.  Specifically,  suppose  that  nodes  can  only 
transmit  to  other  nodes  in  the  same  cell  or  in  adjacent  cells,  and  that  at 
most  one  node  can  transmit  per  cell  during  a  single  timeslot.  Suppose 
that  transmissions  in  adjacent  cells  use  orthogonal  frequency  bands, 
and  that  interference  from  non-adjacent  cells  is  negligible.  In  this  case, 
transmission  decisions  can  be  distributed  node-by-node.  Let  Iab(t)  be 
a  control  process  that  takes  the  value  1  if  link  (a,  b )  is  activated  during 
slot  t.  and  zero  else  (as  in  Example  2.3).  Let  I(t)  =  ( Iab(t ))  represent 
the  matrix  of  transmission  decisions,  restricted  to  the  control  space 
XS{t)  that  specifies  all  feasible  link  activations  under  a  given  topology 
state  S(t).  Suppose  that  the  transmission  rate  of  an  in-cell  transmis¬ 
sion  is  h  packets/slot,  and  that  of  an  adjacent  cell  transmission  is  l 
packets/slot  (where  h  >  l).  The  link  transmission  rate  function  is  thus 
given  by  Cab(I(t),  S(t))  =  Cab(Iab(t),  Sa(t),  Sb(t))  (where  Cab{-)  takes 
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Fig.  2.3  An  ad-hoc  mobile  network  with  a  cell  partitioned  structure. 


units  of  packets/slot),  and  we  have: 

(  h  if  Iab{t)  =  1  and  Sa{t)  =  Sb(t ) 
Cab(Iab(t),Sa(t),Sb(t))  =  <  l  if  Iab(t)  =  1  and  Sa{t)  /  Sb(t)  . 

[  0  else 

where  ( Iab(t ))  £  -^S(t) ■  Similar  cell  partitioned  network  models  are  used 
in  [108]  [117]  [109].  Note  that  this  model  allows  the  possibility  of  a  single 
node  transmitting  over  one  frequency  band  while  simultaneously  receiv¬ 
ing  over  another  frequency  band.  In  systems  where  this  is  infeasible, 
the  additional  constraint  that  a  node  cannot  simultaneously  transmit 
and  receive  must  be  imposed.  This  couples  transmission  decisions  over 
the  entire  network  and  complicates  optimal  distributed  control.  One 
(potentially  sub-optimal)  scheduling  alternative  is  to  randomly  choose 
a  set  of  transmitter  nodes  and  a  set  of  receiver  nodes  every  timeslot 
(as  in  [57]  [122]).  Only  nodes  in  the  receiving  set  are  valid  options  for 
the  transmitters.  Another  approach  is  to  allow  nodes  to  send  trans¬ 
mission  requests ,  and  allow  an  arbiter  to  determine  which  requests 
are  granted.  Several  rounds  of  arbitration  can  take  place  to  improve 
scheduling  decisions.  Simple  types  of  one-step  arbitration  schemes  are 
designed  into  wireless  protocols  such  as  802.11,  where  request  to  send 
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and  clear  to  send  messages  regulate  which  network  links  are  simultane¬ 
ously  active  [125].  Multi-step  arbitration  schemes  are  frequently  used 
in  packet  switches  for  computer  systems  [41]  [3]  [103]  [139].  The  control 
techniques  that  we  develop  in  this  paper  reveal  principled  strategies  for 
making  these  scheduling  decisions  in  terms  of  current  network  condi¬ 
tions  and  desired  performance  objectives. 


These  examples  illustrate  the  wide  class  of  data  networks  that  fall 
within  the  scope  of  our  model.  In  summary,  the  function  C(I(t),  S(t)) 
describes  the  physical  and  multiple  access  layer  properties  of  a  given 
network.2  Viewing  the  network  in  terms  of  this  abstract  function  pro¬ 
vides  insight  into  the  fundamental  control  techniques  applicable  to  all 
data  networks  while  enabling  these  techniques  to  take  maximum  ad¬ 
vantage  of  the  unique  properties  of  each  data  link. 

2.2  Routing  and  Network  Layer  Queueing 

All  data  that  enters  the  network  is  associated  with  a  particular  com¬ 
modity,  which  minimally  defines  the  destination  of  the  data,  but  might 
also  specify  other  information,  such  as  the  source  node  of  the  data  or 
its  priority  service  class.  Let  1C  represent  the  set  of  commodities  in  the 
network,  and  let  K  represent  the  number  of  distinct  commodities  in 
this  set.  Let  A^c\t)  represent  the  amount  of  new  commodity  c  data 
that  exogenously  arrives  to  source  node  i  during  slot  t  (for  all  i  £  N 
and  all  c  £  1C).  We  assume  that  A^c\t)  takes  units  of  bits,  although  it 
can  take  other  units  when  appropriate  (such  as  units  of  packets ).  The 
A^c\t)  data  is  generated  from  the  user  or  application  associated  with 
source  node  i,  and  is  not  necessarily  admitted  directly  to  the  network 
layer.  Rather,  we  view  the  A\c\t)  data  as  arriving  to  the  transport,  layer 
at  node  i,  and  for  each  timeslot  t  we  define  R^c\t)  as  the  amount  of 
commodity  c  data  allowed  to  enter  the  network  layer  from  the  transport 
layer  at  node  i. 

Each  node  i  maintains  a  set  of  internal  queues  for  storing  network 


2  See  [18]  for  a  definition  and  discussion  of  the  various  layers  associated  with  the  standard 
7  layer  Open  Systems  Interconnection  (OSI)  networking  model,  including  the  transport, 
network,  and  physical  layers,  and  the  multiple  access  sub-layer. 


2.2.  Routing  and  Network  Layer  Queueing  19 


Fig.  2.4  A  heterogeneous  network  with  transport  layer  storage  reservoirs  and  internal  net¬ 
work  layer  queues  at  each  node. 


layer  data  according  to  its  commodity  (Fig.  2.4).  Let  u\c\t)  represent 
the  current  backlog,  or  unfinished  work,  of  commodity  c  data  stored  in 
a  network  layer  queue  at  node  i.  The  queue  backlog  U^c\t)  can  con¬ 
tain  both  data  that  arrived  exogenously  from  the  transport  layer  at 
node  i  as  well  as  data  that  arrived  endogenously  through  network  layer 
transmissions  from  other  nodes.  In  the  special  case  when  node  i  is  the 
destination  of  commodity  c  data,  we  formally  define  U^c\t)  to  be  0 
for  all  t,  so  that  any  data  that  is  successfully  delivered  to  its  destina¬ 
tion  is  assumed  to  exit  the  network  layer.  We  assume  that  all  network 
layer  queues  have  infinite  buffer  storage  space.  Our  primary  goal  for 
this  layer  is  to  ensure  that  all  queues  are  stable,  so  that  time  average 
backlog  is  finite.  This  performance  criterion  tends  to  yield  algorithms 
that  also  perform  well  when  network  queues  have  finite  buffers  that  are 
sufficiently  large. 

A  network  layer  control  algorithm  makes  decisions  about  routing, 
scheduling,  and  resource  allocation  in  reaction  to  current  topology 
state  and  queue  backlog  information.  The  resource  allocation  decision 
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m  6  Ism  determines  the  transmission  rates  /iab(t )  =  Cab(I(t),  S(t)) 
offered  over  each  link  (a,  b )  on  timeslot  t.  In  general,  multiple  com¬ 
modities  might  be  transmitted  over  this  link  during  a  single  timeslot.3 
Define  (t)  as  the  rate  offered  to  commodity  c  data  over  the  (a,  b) 
data  link  during  slot  t.  These  /J/^j  (t)  values  represent  routing  decision 
variables  chosen  by  the  network  controller.  It  is  often  convenient  to 
impose  routing  restrictions  for  each  commodity,  and  hence  we  define 
Cc  as  the  set  of  all  links  (a,  b)  that  commodity  c  data  is  allowed  to  use. 
Thus,  the  controller  at  each  node  a  €  M  chooses  the  routing  decision 
variables  (t)  subject  to  the  following  routing  constraints: 

!>$(*)  <M*)>  (2-2) 

c&K 

lJ'ab(t)  =  °>  if  (a,b)$Cc.  (2.3) 

We  assume  that  only  the  data  currently  in  node  i  at  the  beginning 

of  slot  t  can  be  transmitted  during  that  slot.  Hence,  the  slot-to-slot 
dynamics  of  the  queue  backlog  (t)  satisfies  the  following  inequality: 

U\c\t+ l)<max  Ut\t)-J2^\t),0  (2.4) 

b  J  a 

The  above  expression  is  an  inequality  rather  than  an  equality  because 
the  actual  amount  of  commodity  c  data  arriving  to  node  i  during  slot 
t  may  be  less  than  Yla  lloi  (f)  if  t he  neighboring  nodes  have  little  or  no 
commodity  c  data  to  transmit. 

2.2.1  On  the  link  constraint  sets  Cc 

The  routing  constraint  (2.3)  restricts  commodity  c  data  from  using 
links  outside  of  the  set  Cc.  The  constraint  sets  Cc  are  arbitrary,  and 
hence  the  above  model  includes  the  special  case  of  single-hop  networks 
where  only  direct  transmissions  between  nodes  is  allowed.  This  can 
be  accomplished  by  setting  Cc  =  {(a,  b)}  for  each  commodity  c  whose 
traffic  is  originated  at  node  a  and  destined  to  node  b.  Also,  the  above 


We  shall  find  that  we  can  restrict  control  laws  to  transmitting  only  a  single  commodity 
per  link,  without  loss  of  optimality. 
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model  includes  the  special  case  of  unconstrained  routing,  where  each 
set  Cc  contains  all  of  the  links  of  the  network.  In  this  case,  the  network 
does  not  require  a  pre-specified  route.  Routing  decisions  can  be  made 
dynamically  at  each  node,  and  packets  of  the  same  commodity  can 
potentially  traverse  different  paths.  While  unconstrained  routing  allows 
for  the  largest  set  of  options,  it  can  often  be  complex  and  may  lead  to 
large  network  delay  in  cases  when  some  packets  are  transmitted  in 
directions  that  take  them  further  away  from  their  destinations. 

To  ensure  more  predictable  performance  and  to  (potentially)  reduce 
these  delay  problems,  the  link  sets  Cc  can  be  designed  in  advance  to 
ensure  that  all  transmissions  move  commodities  closer  to  their  destina¬ 
tions.  Note  that  restricting  the  routing  options  makes  the  network  less 
capable  of  adapting  to  random  link  failures,  outages,  or  user  mobility, 
whereas  unconstrained  routing  can  in  principle  adapt  by  dynamically 
choosing  a  new  direction. 

Both  unconstrained  and  constrained  routing  allow  for  a  multiplicity 
of  paths.  In  cases  when  it  is  desirable  to  restrict  sessions  to  a  single 
path  (perhaps  to  ensure  in-order  packet  delivery),  each  set  Cc  can  be 
specified  as  a  directed  tree  with  final  node  given  by  the  destination 
node  for  commodity  c.  Alternatively,  in  cases  when  it  is  desired  for 
different  paths  to  cross  but  not  merge,  a  different  commodity  c  can  be 
associated  with  each  different  source-destination  pair,  and  the  link  set 
Cc  is  defined  as  the  set  of  all  links  in  the  path  for  commodity  c. 

2.3  Flow  Control  and  the  Transport  Layer 

All  exogenous  arrivals  A^c\t)  first  enter  the  transport  layer  at  their 
corresponding  source  nodes,  and  this  data  is  held  in  storage  reservoirs 
to  await  acceptance  to  the  network  layer  (Fig.  2.4).  We  assume  there 
is  a  separate  storage  reservoir  for  each  commodity  at  each  node,  and 
define  L^c\t )  as  the  backlog  of  commodity  c  bits  currently  stored  in 
the  transport  layer  storage  reservoir  at  node  i.  Every  timeslot,  each 
source  node  i  makes  flow  control  decisions  by  choosing  the  amount  of 
bits  RjC\t)  to  deliver  to  the  network  layer  at  node  i,  subject  to  the 
constraint  Ft\c\t)  <  L\\t)  +  A^\t)  for  all  ( i,c )  and  all  t,  and  subject 
to  some  additional  constraints  made  precise  in  Chapter  5. 
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The  storage  reservoirs  for  each  commodity  may  be  infinite  or  finite, 
with  size  0  <  L rTax  <  oo.  Therefore,  some  data  must  be  dropped  if 
the  new  exogenous  arrivals  are  not  admitted  to  the  network  layer  and 
do  not  fit  into  the  storage  reservoir.  Hence,  L[c\t)  <  LrjrLax  for  all  t, 
and  the  dynamics  of  storage  buffer  (z,  c)  from  one  timeslot  to  the  next 
satisfies  the  following  inequality: 


L^it  +  1)  <  min  Lf’(t)  -  Rf\t)  +  L 


■(c) 


(c)/ 


I  (c)/ 


The  reason  that  the  above  expression  is  an  inequality  (rather  than  an 
equality) ,  is  that  the  amount  of  bits  to  drop  is  chosen  arbitrarily  by  the 
flow  controller,  and  in  particular  the  controller  might  decide  to  drop 
all  bits  associated  with  a  particular  packet  in  the  case  when  a  com¬ 
plete  packet  does  not  fit  into  the  storage  reservoir.  The  storage  buffer 
size  is  arbitrary,  possibly  zero.  In  the  case  when  LT1J1X  =  0,  all 

data  that  is  not  immediately  admitted  to  the  network  layer  is  neces¬ 
sarily  dropped.  In  cases  when  >  0,  the  flow  controller  must  make 

additional  decisions  about  which  data  to  drop  whenever  appropriate. 

In  Chapters  3-4  we  shall  find  it  useful  to  neglect  flow  control  de¬ 
cisions  entirely,  so  that  all  arriving  data  is  immediately  admitted  to 
the  network  layer  and  R^c\t)  =  A^c\t)  for  all  timeslots  t.  In  this  case 
we  say  the  flow  controllers  are  “turned  off.”  This  action  of  “turning 
off’  the  flow  controllers  is  only  used  as  a  thought  experiment  to  build 
understanding  of  network  layer  routing  and  stability  issues.  In  prac¬ 
tice,  turning  off  the  flow  controllers  can  lead  to  instability  problems 
in  cases  when  network  traffic  exceeds  network  capabilities,  and  these 
issues  are  considered  in  detail  in  Sections  5-6  when  flow  control  is  again 
integrated  into  the  problem  formulation. 


2.4  Discussion  of  the  Assumptions 

In  this  section  we  discuss  the  assumptions  stated  previously  about  the 
network  model  and  its  mode  of  operation. 


2.4.1  The  Time  Slot  Assumption 

Timeslots  are  used  to  facilitate  analysis  and  to  cleanly  represent  periods 
corresponding  to  new  channel  conditions  and  control  actions.  However, 
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this  assumption  presumes  synchronous  operation,  where  control  actions 
throughout  the  network  take  place  according  to  a  common  timeclock. 
Although  asynchronous  networking  is  not  formally  considered  in  this 
paper,  the  timescale  expansion  and  approximate  scheduling  results  of 
[108]  [134]  [122]  suggest  that  the  algorithms  and  analysis  developed  here 
can  be  extended  to  systems  with  independent  network  components  that 
operate  on  their  own  timescales.  Asynchronous  systems  are  further  ex¬ 
plored  in  [26]. 

The  assumption  that  channels  hold  their  states  for  the  duration  of 
a  timeslot  is  clearly  an  approximation,  as  real  physical  systems  do  not 
conform  to  fixed  slot  boundaries  and  may  change  continuously.  This 
approximation  is  valid  in  cases  where  slots  are  short  in  comparison  to 
the  speed  of  channel  variation.  In  a  wireless  system  with  predictable 
slow  fading  and  non-predictable  fast  fading  [23]  [105],  the  timeslot  is 
assumed  short  in  comparison  to  the  slow  fading  (so  that  a  given  mea¬ 
surement  or  prediction  of  the  fade  state  lasts  throughout  the  timeslot) 
and  long  in  comparison  to  the  fast  fading  (so  that  a  transmission  of 
many  symbols  encoded  with  knowledge  of  the  slow  fade  state  and  the 
fast-fade  statistics  can  be  successfully  decoded  with  sufficiently  low  er¬ 
ror  probability). 

2.4.2  Channel  Measurement 

We  assume  that  network  components  have  the  ability  to  monitor  chan¬ 
nel  quality  so  that  intelligent  control  decisions  can  be  made.  This  mea¬ 
surement  can  be  in  the  form  of  a  specific  set  of  attenuation  coefficients, 
or  can  be  according  to  a  simple  channel  classification  such  as  “Good,” 
“Medium,  “Bad.”  Channel  measurement  technology  is  currently  being 
implemented  for  cellular  communication  with  High  Data  Rate  (HDR) 
services  [63] ,  and  the  ability  to  measure  and  react  to  channel  informa¬ 
tion  is  expected  to  improve  significantly.4  In  systems  where  it  is  diffi¬ 
cult  to  obtain  timely  feedback  about  channel  quality,  such  as  satellite 
systems  with  long  round-trip  times,  channel  measurement  can  be  com¬ 
bined  with  channel  prediction.  Accurate  channel  prediction  schemes  for 

4  Indeed,  it  is  claimed  in  [152]  that  channel  measurements  can  be  obtained  almost  as  often 
as  the  symbol  rate  of  the  link  in  certain  local  area  wireless  networks. 
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satellites  are  developed  in  [32]  [33]  [69]. 

2.4.3  The  Error- Free  Transmission  Assumption 

All  data  transmissions  from  one  node  to  the  next  are  considered  to  be 
successful  with  sufficiently  high  probability.  For  example,  the  link  bud¬ 
get  curves  for  wireless  transmissions  could  be  designed  so  that  decoding 
errors  occur  with  probability  less  than  10~6.  In  such  a  system,  there 
must  be  some  form  of  error  recovery  protocol  which  allows  a  source 
to  re-inject  lost  data  back  into  the  network  [18].  If  transmission  errors 
are  rare,  the  extra  arrival  rate  due  to  such  errors  is  small  and  does  not 
appreciably  change  network  performance.  Throughout  this  paper,  we 
neglect  such  errors  and  treat  all  transmissions  as  if  they  are  error- free. 
An  alternate  model  in  which  transmissions  are  successful  with  a  given 
probability  can  likely  be  treated  using  similar  analysis.  Recent  work 
in  [75]  [74]  considers  channel  uncertainty  for  transmission  scheduling 
in  MIMO  systems,  and  work  in  [113]  considers  routing  in  multi-hop 
networks  with  unreliable  channels  and  and  multi-receiver  diversity. 
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Stability  and  Network  Capacity 


Here  we  establish  the  fundamental  throughput  limitations  of  a  general 
multi-commodity  network  as  defined  in  the  previous  section.  Specif¬ 
ically,  we  characterize  the  network  layer  capacity  region.  This  region 
describes  the  set  of  traffic  rates  that  the  network  can  stably  support, 
considering  all  possible  strategies  for  choosing  the  control  decision  vari¬ 
ables  that  affect  routing,  scheduling,  and  resource  allocation.  We  begin 
with  a  precise  definition  of  stability  for  single  queues  and  for  queueing 
networks. 

3.1  Queue  Stability 

Consider  a  single  queue  with  an  input  process  A(t)  and  transmission 
rate  process  where  A(t)  represents  the  amount  of  new  arrivals  that 
enter  the  queue  during  slot  t,  and  fi(t)  represents  the  transmission  rate 
of  the  server  during  slot  t.  We  assume  that  the  A(t)  arrivals  occur  at 
the  end  of  slot  t,  so  that  they  cannot  be  transmitted  during  that  slot. 
Let  U (t)  represent  the  current  backlog  in  the  queue.  The  U (t)  process 
evolves  according  to  the  following  discrete  time  queueing  law: 

U(t  +  1)  =  max[H (t)  —  n(t),  0]  +  A(t). 

The  queue  might  be  located  within  a  larger  network,  in  which  case 
the  arrival  process  A(t)  is  composed  of  random  exogenous  arrivals  as 
well  as  endogenous  arrivals  resulting  from  routing  and  transmission 
decisions  from  other  nodes  of  the  network.  Likewise,  the  transmission 
rate  ji{t)  can  be  determined  by  a  combination  of  random  channel  state 
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variations  and  controlled  network  resource  allocations,  both  of  which 
can  change  from  slot  to  slot.  Therefore,  it  is  important  to  develop  a 
general  definition  of  queueing  stability  that  handles  arbitrary  A(t)  and 
fi(t)  processes. 

Definition  3.1.  A  queue  is  called  strongly  stable  if: 

1  4-1 

limsup  -  ^2  IE  <  oo. 

t— >00  t  n 

T= 0 


That  is,  a  queue  is  strongly  stable  if  it  has  a  bounded  time  average 
backlog.  This  leads  to  a  natural  definition  of  network  stability: 

Definition  3.2.  A  network  is  strongly  stable  if  all  individual  queues 
of  the  network  are  strongly  stable. 


A  discussion  of  more  general  stability  definitions  can  be  found  in 
[108]  [122]  [58]  [42]  [12].  Throughout  this  paper  we  shall  restrict  at¬ 
tention  to  the  strong  stability  definition  given  above,  and  shall  often 
use  the  term  “stability”  to  refer  to  strong  stability.  The  following  sim¬ 
ple  but  important  necessary  condition  holds  for  strongly  stable  queues 
with  any  arbitrary  arrival  and  server  processes  (possibly  without  well 
defined  time  averages).  Its  proof  can  be  found  in  [110]. 


Lemma  3.3.  (Necessary  Condition  for  Strong  Stability)  If  a  queue 
is  strongly  stable  and  either  IE{A(f)}  <  Amax  for  all  t,  or 
IE{/i(f)  —  A(t)}  <  Jmay  for  all  t,  where  Amax,  Dmax  are  finite  non¬ 
negative  constants,  then: 


lim 

t — >oo 


®{um 

t 


=  0. 


(3.1) 


3.1.1  The  Arrival  Process  Assumptions 

To  analyze  network  capacity,  we  assume  that  all  exogenous  arrival 
processes  A^c\t)  satisfy  the  following  structural  properties  for  admis¬ 
sible  inputs. 
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Definition  3.4.  An  arrival  process  A(t )  is  admissible  with  rate  A  if: 

•  The  time  average  expected  arrival  rate  satisfies: 

1  t_1 

lim  -^E{A(r)}  =  A. 

t—>oc  t  z ' 

T— 0 

•  There  exists  a  finite  value  A.max  such  that  IE  |A(f)2  | H  ( t )  }  < 
A-max  f°r  all  timeslots  t ,  where  H  ( t )  represents  the  history 
up  to  time  t ,  i.e.,  all  events  that  take  place  during  slots  r  6 
{0, ... -  1}. 

•  For  any  5  >  0,  there  exists  an  interval  size  T  (that  may 
depend  on  5 )  such  that  for  any  initial  time  to  the  following 
condition  holds: 

E<f^E^o  +  A0|H(to)}  <  X  +  S.  (3.2) 

l  1  k= 0  J 


Some  examples  of  admissible  arrival  processes  are  the  following. 

Example  3.1.  Let  X  ( t )  be  an  ergodic  Markov  chain  with  a  finite 
state  space  {1, . . . ,  Q}.  When  X  ( t )  =  m,  let  A(t)  be  chosen  indepen¬ 
dently  and  identically  distributed  (i.i.d.)  with  distribution  P^\a). 
If  7rm,  m  £  {1  is  the  steady  state  distribution  of  X  (t)  and 

E{A(t)  \X  ( t )  =  m}  =  Am,  then  the  process  A  (t)  is  admissible  with 
rate  A  =  Ylm= l  ^mXm-  An  important  special  case  is  when  there  is  only 
one  state,  so  that  A(t)  is  i.i.d.  every  slot  with  lE{A(f)}  =  A  for  all  t. 


Example  3.2.  Let  A  (f)  satisfy  the  following  burstiness  constraints 
t2-i 

A  (£2  -  h)  +  cti  >  ^  A  (t)  >  A  (f2  -  h)  -  cr2,  for  all  t2  >  1 1  >  0  , 

t=t\ 

where  cj\  and  o2  are  nonnegative  numbers.  Then  A  ( t )  is  admissible 
with  rate  A.  Burstiness  constrained  models  have  been  used  extensively 
in  wired  networks  [36],  [143],  [27],  [82]  and  in  a  wireless  context  in  [159]. 
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Below  we  define  the  concept  of  an  admissible  service  process  jt(t): 

Definition  3.5.  A  server  process  jt(t)  is  admissible  with  time  average 
service  rate  JL  if: 

•  The  time  average  expected  service  rate  satisfies: 

1  t_1 

lirn  -^E{g(r)}  =  l-i ■ 

t — >oo  t  z ' 

T= 0 

•  There  exists  an  upper  bound  jimax  such  that  ji(t)  <  jtmax  for 
all  t. 

•  For  any  5  >  0,  there  exists  an  interval  size  T  (that  may 
depend  on  (5),  such  that  for  any  initial  time  to  the  following 
condition  holds: 

+  |H(*o)l  >p-<5.  (3-3) 

l  1  k= 0  J 


Lemma  3.6.  (Stability  Conditions  under  Admissibility)  Consider  a 
queue  with  an  admissible  input  process  A(t)  with  arrival  rate  A,  and 
an  admissible  server  process  with  time  average  rate  JL.  Then:  (a)  A  <  JL 
is  a  necessary  condition  for  strong  stability,  (b)  A  <  /x  is  a  sufficient 
condition  for  strong  stability. 


The  necessary  condition  is  quite  intuitive.  Indeed,  if  A  >  Jl,  then 
expected  queue  backlog  necessarily  grows  to  infinity,  leading  to  insta¬ 
bility.  The  sufficient  condition  is  also  intuitive,  but  its  proof  requires  the 
structure  of  admissible  arrival  and  service  processes  as  defined  above 
(see  [108]  for  a  proof).  We  note  that  strong  stability  also  holds  in  cases 
when  the  infinite  time  average  conditions  for  A(t )  and  jt(t)  do  not  neces¬ 
sarily  hold,  but  these  processes  satisfy  all  other  inequality  conditions  of 
the  admissibility  definitions  (for  some  values  A  and  JL  such  that  A  <  JL) . 
We  say  that  such  an  arrival  process  is  admissible  with  arrival  rate  less 
than  or  equal  to  A,  and  such  a  service  process  is  admissible  with  average 
service  rate  greater  than  or  equal  to  Jl. 
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3.2  The  Network  Layer  Capacity  Region 

Consider  a  network  with  a  general  link  transmission  rate  matrix 
C(I(t),  S(t))  =  ( Cob(I(t ),  S(t))).  Recall  that  I(t)  €  %s(t)  and  S(t)  €  S , 
where  2 is  the  control  space  for  a  given  topology  state  S(t),  and  S  is 
the  finite  set  of  all  possible  topology  states  for  the  network.  The  func¬ 
tion  C(-,-)  is  arbitrary  (possibly  discontinuous)  and  is  only  assumed 
to  be  bounded,  so  that  for  all  links  (a,  b )  and  all  /6ls  and  s  €  S  we 
have: 

0  f;  Cfafe(/,  s)  <  Rmax-  (3-4) 

for  some  maximum  transmission  rate  Umax-  The  topology  state  S(t)  is 
assumed  to  evolve  according  to  a  finite  state,  irreducible  Markov  chain 
(possibly  periodic) .  Such  chains  have  well  defined  time  averages  irs ,  rep¬ 
resenting  the  time  average  fraction  of  time  that  S(t)  =  s.  Specifically, 
with  probability  1  we  have:  1 

1 

fhiu  -  ^2  l[S(t)=s]  =  ns  ,  for  all  s  G  S.  (3.5) 

T  — 0 

where  1  [.S'(^)=s]  an  indicator  function  that  takes  the  value  1  whenever 
S(t)  =  s,  and  takes  the  value  zero  otherwise. 

Let  J\f  and  K.  represent  the  set  of  nodes  and  commodities,  with 
sizes  N  and  K  respectively.  Let  Cc  be  the  set  of  links  defining  the 
routing  constraints  for  each  commodity  c  £  /C.  Define  U-C\t )  as  the 
internal  queue  backlog  of  commodity  c  data  at  node  i.  Due  to  the  rout¬ 
ing  constraints,  some  commodities  might  never  be  able  to  visit  certain 
nodes.  Further,  some  nodes  might  only  be  associated  with  destina¬ 
tions,  and  hence  these  nodes  do  not  keep  any  internal  queues.  Hence, 
we  define  Kj  as  the  number  of  internal  queues  kept  by  node  i.  where 
Ki  6  {0, 1, . . . ,  K }.  Define  V  as  the  set  of  all  node-commodity  pairs 
(i,  c)  associated  with  internal  queues  in  the  network,  and  let  D  repre¬ 
sent  the  number  of  such  queues: 

N 

d=Y.k<- 

i=  1 


1  The  Markov  structure  for  S(t)  is  used  only  to  facilitate  presentation.  Our  results  hold  more 
generally  for  any  S(t)  that  satisfies  the  channel  convergent  property  defined  in  [108]. 
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The  integer  D  defines  the  relative  dimension  of  the  network. 

Let  A^c\t)  represent  the  process  of  commodity  c  bits  arriving  ex¬ 
ogenously  to  node  i.  Assume  that  these  arrival  processes  are  admissible 
with  time  average  rates  A^  (in  units  of  bits/slot),  and  let  A  =  (A^) 
represent  the  arrival  rate  matrix.  We  assume  throughout  that  A-  =  0 
whenever  (i,  c)  ^  V,  so  that  the  number  of  non-zero  rates  is  less  than 
or  equal  to  D.  Assume  that  the  network  flow  controllers  are  turned 
off,  so  that  all  incoming  traffic  arrives  directly  to  the  network  layer. 


Definition  3.7.  The  network  layer  capacity  region  A  is  the  closure  of 
the  set  of  all  arrival  rate  matrices  ^A-c^  that  can  be  stably  supported 
by  the  network,  considering  all  possible  strategies  for  choosing  the 
control  variables  to  affect  routing,  scheduling,  and  resource  alloca¬ 
tion  (including  strategies  that  have  perfect  knowledge  of  future  events). 


Note  that  this  is  a  network  layer  notion  of  capacity  that  considers  all 
choices  of  the  decision  variables  (t)  and  I  (t)  for  a  network  that  op¬ 
erates  according  to  a  given  C(I(t),  S(t))  function;  it  was  introduced  in 
[142]  [147]  and  generalized  further  in  [144]  [108]  [122],  This  is  distinct 
from  the  information  theoretic  notion  of  network  capacity,  which  in¬ 
cludes  optimization  over  all  possible  modulation  and  coding  strategies 
and  involves  many  of  the  unsolved  problems  of  network  information 
theory  [34],  The  network  layer  and  information  theoretic  capacity  re¬ 
gions  are  called  "stability"  and  "capacity"  regions  respectively  in  [99], 
where  a  third  notion  of  "throughput  region"  referring  to  the  case  when 
all  nodes  have  infinite  backlogs,  is  also  examined.  In  this  work,  we  shall 
use  the  term  "capacity  region"  to  refer  to  the  network  layer  capacity 
region  as  described  above. 

The  issue  of  capacity  scaling  was  raised  in  [59]  where  it  was  found 
that  the  capacity  vanishes  asymptotically  as  the  number  of  nodes  in¬ 
creases,  with  a  rate  that  is  inversely  proportional  to  a  fractional  power 
of  the  number  of  nodes.  The  type  of  capacity  considered  in  [59]  is 
similar  to  the  one  we  consider  here.  More  specifically  a  certain  model 
is  considered  for  the  local  interaction  of  the  radio  transmissions,  that 
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imply  interference  restrictions  on  simultaneous  transmissions  of  nodes 
with  geographical  proximity.  Assuming  a  uniform  end-to-end  traffic 
load  matrix  then,  the  capacity  can  be  specified  by  a  scalar,  i.e.  the 
maximum  traffic  intensity  that  is  sustainable  under  any  transmission 
control  and  traffic  forwarding  policy.  In  [59]  a  bound  to  that  capacity 
is  obtained  that  vanishes  inversely  proportional  to  the  square  root  of 
the  number  of  nodes.  That  important  results  indicates  that  large  scale 
ad-hoc  wireless  networks  with  flat  architecture  may  only  have  limited 
usability  as  a  general  purpose  communication  infrastructure.  It  moti¬ 
vated  a  lot  of  follow  up  work  generalizing  the  result  in  various  ways, 
some  indicative  examples  are  [60]  [57]  [78]  [95]  [116]  [151]  [166]  [91] 

[117]- 

3.2.1  Constructing  the  Capacity  Region 

To  build  intuition  about  the  set  A,  we  first  consider  the  capacity  region 
of  a  traditional  wireline  network  with  no  time  variation,  such  as  the 
static  network  of  Example  2.1  in  Chapter  2.  Such  a  network  is  described 
by  a  constant  matrix  ( Gab ),  where  Gab  is  the  fixed  rate  at  which  data 
can  be  transferred  over  link  (a,  6),  and  Gab  =  0  if  there  is  no  physical 
link  from  node  a  to  node  b.  The  network  capacity  region  in  this  case 
is  described  implicitly  as  the  set  of  all  arrival  rate  matrices  for 

which  there  exist  multi- commodity  flow  variables  (for  a,b  £  N 
and  c  €  1C)  that  satisfy  a  set  of  flow  conservation  equations,  and  that 
additionally  satisfy  the  link  constraint  Ylc  <  Gab  for  all  links  (a,  b). 
This  constraint  ensures  that  the  total  flow  over  any  link  does  not  exceed 
the  link  transmission  rate.  Intuitively,  this  coincides  with  the  necessary 
and  sufficient  conditions  for  queue  stability  described  in  Lemma  3.6. 
Indeed,  stability  requires  that  the  data  arrival  rate  to  any  link  is  no 
more  than  the  transmission  rate  of  the  link,  regardless  of  whether  data 
flows  as  a  continuous  fluid  or  as  packetized  units. 

The  capacity  region  of  a  general  network  differs  from  that  of  a  static 
wireline  network  only  in  the  description  of  the  link  constraint.  Indeed, 
first  note  that  the  time  varying  network  topology  requires  link  trans¬ 
mission  capabilities  to  be  defined  in  a  time  average  sense,  where  the 
resulting  transmission  rate  over  a  given  link  (a,  b)  is  averaged  over  all 
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possible  topology  states.  Second,  the  resulting  time  average  link  rates 
are  not  fixed,  but  depend  on  the  resource  allocation  policy  for  choosing 
I(t)  €  XS{ty  Thus,  instead  of  describing  the  network  as  a  single  graph 
with  a  single  transmission  rate  matrix  (Ga&),  the  network  is  described 
by  a  collection  of  graphs  defined  by  a  graph  family  F.  The  graph  family 
T  can  be  viewed  as  the  set  of  all  long-term  transmission  rate  matrices 
(Gab)  that  the  network  can  be  configured  to  support  on  the  single-hop 
links  connecting  node  pairs  (a,  b),  and  is  defined  as  follows: 

T=  7 TsConv {C(I,  s)  \I  gls},  (3-6) 

sGcS 

where  addition  and  scalar  multiplication  of  sets  is  used2,  and  where 
Conv  {.4}  represents  the  convex  hull  of  the  set  A.  Specifically, 
Conv  {.4,}  is  defined  as  the  set  of  all  finite  weighted  combinations 
P\a\  T  P2&2  +  •  •  •  +  Pmam  of  elements  £  A  (where  {pi}  are  nonneg¬ 
ative  numbers  summing  to  1).  Such  weighted  combinations  are  called 
convex  combinations.  To  intuitively  understand  why  the  graph  family 
T  has  the  form  given  in  (3.6),  we  note  the  following  basic  result  from 
convex  set  theory: 

Fact  1:  If  fi  is  any  random  vector  that  takes  values  within  some 
general  set  A.  then  IE  {/x}  £  Conv  {.4}  (assuming  the  expectation  is 
well  defined).  □ 

Consider  now  the  set  As  defined  as  the  set  of  all  transmission  rate 
matrices  possible  under  channel  state  s: 

AS={C(I,  s)  \I  £  ls }. 

Two  example  sets  As,  corresponding  to  two  different  topology  states, 
are  shown  in  Fig.  3.1.  Suppose  we  have  a  resource  allocation  algo¬ 
rithm  that  chooses  a  random  control  input  I(t )  £  Zs  according  to 
some  probability  law  whenever  S(t )  =  s,  yielding  a  random  rate  ma¬ 
trix  pt(t)  =  C(I(t),s).  By  definition,  this  random  rate  matrix  satisfies 
fi(t )  £  As,  and  hence  by  Fact  1  it  follows  that  the  expected  rate  ma¬ 
trix  satisfies:  E {/x(t)  IS'(t)  =  s}  £  Conv{As}-  Thus,  randomizing  the 

2  For  vector  sets  A.B  and  scalars  a,/3,  the  set  aA  +  /3B  is  defined  as  {7  |  7  =  aa  +  /3b  : 
for  some  a  £  A,b  6  B}. 
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Fig.  3.1  A  construction  of  the  set  F  for  the  case  of  2  dimensions,  illustrating  the  set  of 
all  achievable  long  term  link  rates  (^1,^2).  In  this  example,  we  consider  only  two  chan¬ 
nel  states  si  and  S2,each  equally  probable.  Note  that  for  the  first  channel  state,  the  set 
{C  (/,  s)  \I  Gls}  is  disconnected  and  non-convex.  Its  convex  hull  is  shown  in  the  first  plot. 
The  secont  plot  illustrates  the  weighted  sum  of  the  convex  hull  of  the  regions  associated 
with  each  of  the  two  channel  states.  This  is  the  F  region,  and  is  necessarily  convex. 

control  decisions  allows  the  expected  rate  matrix  to  expand  beyond  the 
set  „4S  to  reach  points  within  the  larger  set  Conv{As}  (see  Fig.  3.1). 
Further,  by  appropriately  choosing  the  randomized  probabilities,  any 
point  within  the  set  T  can  be  reached  in  this  way.  This  is  summarized 
in  the  following  fact. 

Fact  2:  A  matrix  G  =  ( Gab )  is  in  the  graph  family  T  if  and  only 
if  there  exists  a  randomized  policy  that  bases  control  decisions  on  the 
current  channel  state,  such  that: 


(3.7) 


where  E{/i(t)  \S(t)  =  s}  is  the  expected  rate  matrix  offered  by  the 
randomized  policy  when  S(t)  =  s.  □ 

By  ergodicity  of  the  topology  state  process  S(t )  together  with 
the  law  of  large  numbers,  it  is  easy  to  see  that  the  right  hand 
side  of  (3.7)  is  an  expression  for  the  time  average  transmission  rate 
lim^oo  |  At(7")-  Thus,  the  network  can  be  configured  to  achieve 

long  term  link  transmission  rates  for  any  rate  matrix  G  within  the 
graph  family  T.  Let  Cl{T}  represent  the  closure  of  the  set  T.  The  fol¬ 
lowing  theorem  from  [108]  [122]  characterizes  the  network  layer  capac- 
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ity  region.  Recall  that  every  timeslot,  a  network  controller  must  choose 
m  e  and  must  choose  routing  variables  (t)  that  satisfy: 

<  Cab(I(t),S(t)),  =  0  if  (a, b)  Cc. 


Theorem  3.8.  (Capacity  Region  for  a  Network)  The  capacity  region 
of  the  network  is  given  by  the  set  A  consisting  of  all  input  rate  matrices 
such  that  A-c^  =  0  whenever  (■ i,c )  ^  V,  and  such  that  there 
exists  a  rate  matrix  (Gab)  €  Cl{T}  together  with  multi-commodity 
flow  variables  |/^|  satisfying: 

(Flow  Efficiency  Constraints) 

fab  >  0,  f$  =  fdest(c),b  =  °>  for  a11  a,bcAf,cC)C,  (3.8) 

(Flow  Conservation  Constraints) 

ASC)  =  fib  ~  fa?’  for  a11  (*’ c)  G  V  with  *  /  dest(c),  (3.9) 

b  a 


(Routing  Constraints) 

=  0,  for  all  a,  b  €  A f,  c  €  1C  with  (a,  b)  ^  Cc,  (3.10) 

(Link  Constraints) 

^f^<Gab,  for  all  a,  6  G  AA,  (3.11) 

C 

where  dest(c)  represents  the  destination  node  for  commodity  c  data. 


Thus,  a  rate  matrix  ^A-c^  is  in  the  capacity  region  A  if  there  exists 
a  matrix  (Gab)  £  C7{r}  that  defines  link  capacities  in  a  traditional 
graph  network,  such  that  there  exist  multi-commodity  flow  variables 
|  |  which  support  the  ^A-c^  rates  with  respect  to  this  graph.  Note 

that  inequalities  (3.8)  constrain  flow  variables  to  be  non-negative  and 
to  be  “efficient,”  in  that  no  node  transmits  data  to  itself  and  no  node 
re-injects  delivered  data  back  into  the  network.  Inequality  (3.9)  is  a 


3.2.  The  Network  Layer  Capacity  Region  35 


conservation  constraint  that  ensures  the  total  flow  of  commodity  c  data 
into  a  given  node  i  is  less  than  or  equal  to  the  total  flow  out  of  that 
node,  provided  that  node  i  is  not  the  destination.  We  note  that  (3.9) 
is  expressed  as  an  equality  constraint  only  to  facilitate  understanding. 
The  same  theorem  holds  if  (3.9)  is  replaced  by  the  following  inequality 
constraint: 

A-c)  <  Y  fib'’  ~  Y  fai  for  a11  (*>  c)  G  with  i  /  dest(c). 

b  a 

The  above  constraint  is  more  useful  because  it  leads  to  a  simpler  proof 
of  the  theorem  (see  [108]  [122]),  and  also  simplifies  construction  of  dual 
algorithms  for  finding  the  multi-commodity  flows  |/^|  in  the  case 
when  the  set  C7{T}  is  known  in  advance  and  the  problem  is  treated  as 
a  convex  program  (see  Section  4.10). 

The  following  useful  corollary  establishes  an  important  property  of 
the  capacity  region: 

Corollary  3.9.  If  T  =  C7(T)  and  if  the  topology  state  S(t)  is  i.i.d. 
from  slot  to  slot,  then  a  rate  matrix  is  within  the  capacity  region 

A  if  and  only  if  there  exists  a  stationary  randomized  control  algorithm 
that  makes  valid  (t)  decisions  based  only  on  the  current  topology 
state  S(t),  and  that  yields  for  all  (i,  c)  and  all  time  t: 

®  I  Y  dbW  -  Pah1)  f  =  ASC)’  for  a11  (*»  c)  ev  with  i  /  dest  (c) , 

l  b  a  ) 

where  the  expectation  is  taken  with  respect  to  the  random  topology 
state  S(t)  and  the  (potentially)  random  control  action  based  on  this 
state. 

The  constraints  of  Theorem  3.8  lead  to  the  following  structural 
properties  for  A: 

•  The  set  A  is  convex,  closed,  and  bounded  [108]  [19]. 

•  The  set  A  contains  the  all-zero  matrix  (so  that  0  £  A). 

•  If  A  G  A,  then  A  6  A,  where  A  is  any  rate  matrix  that  is 
entrywise  less  than  or  equal  to  A. 
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Let  D  represent  the  subset  of  V  consisting  of  all  node-commodity 
pairs  ( i ,  c)  for  which  it  is  possible  to  stably  support  a  non-zero  input 
rate  A.-^  (assuming  that  all  other  input  rates  are  zero).  The  relative 
interior  of  the  set  A  is  given  by  the  set  of  all  rate  matrices  A  for  which 
there  exists  an  e  >  0  such  that  A  +  e  £  A,  where  e  is  a  matrix  with 
entries  ef^  =  e  for  all  (i,  c)  6  T>,  and  all  other  entries  equal  to  zero.  The 
proof  of  Theorem  3.8  involves  showing  that  ^A?-c^  £  A  is  necessary 

for  stability,  and  that  ^A^c^  within  the  relative  interior  of  A  is  suffi¬ 
cient.  Note  that  although  the  exogenous  arrival  processes  are  assumed 
to  be  admissible,  the  capacity  region  must  capture  all  possible  routing, 
scheduling,  and  resource  allocation  strategies ,  including  strategies  that 
result  in  non-admissible  arrival  or  service  processes  at  the  individual 
queues  of  the  network.  In  the  next  section,  we  prove  the  sufficient  con¬ 
ditions  of  Theorem  3.8  for  the  special  case  of  a  one-hop  network.  For 
the  complete  proof  for  the  general  multi-hop  case,  the  reader  is  referred 
to  [108]  [122], 

3.3  The  Capacity  of  One  Hop  Networks 

Consider  the  special  case  of  a  one-hop  network  with  L  different  exoge¬ 
nous  input  processes.  For  simplicity,  assume  that  data  from  each  input 
process  is  intended  for  transmission  over  a  unique  link.  Let  /r(t)  = 
(/ii(f), . . . ,  HL(t))  represent  the  vector  of  link  transmission  rates  during 
slot  t,  where  Hi{t)  =  Ci(I(t),  S(t))  denotes  the  rate  over  link  l  under 
control  input  I(t)  and  topology  state  S(t).  The  corresponding  graph 
family  T  defined  in  (3.6)  is  thus  a  set  of  rate  vectors  rather  than  rate 
matrices.  Let  A(t)  =  (Ai(t), . . . ,  Ai,(t ))  represent  the  vector  of  exoge¬ 
nous  arrivals,  where  Ai(t)  is  the  number  of  bits  that  arrive  to  link  l 
during  slot  t  (for  l  £  {1, . . . ,  L}).  Assume  these  processes  are  admissi¬ 
ble  with  rate  vector  A  =  (Ai, . . . ,  A l)-  We  have  the  following  corollary 
to  Theorem  3.8. 


Corollary  3.10.  (Single  Hop  Capacity  Region)  The  single  hop  capac¬ 
ity  region  A  consists  of  all  rate  vectors  A  =  (Ai, . . . ,  \l)  such  that  there 
exists  a  vector  (G\, . . . ,  Gl)  £  C7{r}  such  that  \  <  Gi  for  all  network 


3.3.  The  Capacity  of  One  Hop  Networks  37 


links  l  £  { 1 , . . . ,  L} . 


We  skip  the  proof  of  necessity  of  this  corollary  and  concentrate 
on  the  proof  of  sufficiency  since  this  is  important  for  the  subsequent 
development  and  demonstrates  the  design  issues  involved. 

Proof.  (Sufficiency)  Suppose  for  simplicity  that  A  has  full  dimension. 
Suppose  each  link  l  £  {1 , ,L}  receives  an  admissible  input  process 
of  rate  A;.  Let  A  represent  the  input  rate  vector.  Assume  that  A  is  in 
the  interior  of  the  set  A  defined  in  Corollary  3.10,  so  that  there  exists 
an  e  >  0  such  that  (Ai  +  e, . . . ,  Al  +  e)  £  A.  Thus,  there  exists  a  vector 
G  =  (Gi, . . . ,  Gl)  such  that  G  £  Cl{T}  and  A;  +  e  <  Gi  for  all  links 
l  £  {1, ,  L}.  Then  there  must  exist  a  matrix  G  =  [G i, . . . ,  Gl)  such 
that  G  £  T  and  A i  +  e/2  <  Gi  for  all  links  l.  Because  G  £  T,  it  can  be 
written  as: 

G  =  ^nsGs,  (3.12) 

sE«S 

where  Gs  £  Conv{C(I,  s)  \ I  6ls}  for  each  channel  state  s  £  S.  Fur¬ 
thermore,  by  Caratheodory’s  Theorem  [19],  each  matrix  Gs  can  be 
decomposed  into  a  convex  combination  of  at  most  L  +  1  elements  of 

{c(i,s)\i  £isy. 

Gs  =  p\C{ll  s)  +p2sC(ll  s)  +  .. .  +^+1C(/si+1, 8),  (3.13) 

where  Ps  £  Xs  for  all  s  £  S  and  i  £  {1, . . . ,  L  +  1}.  Given  such  a 
decomposition  for  each  channel  state  s  £  S,  the  following  control  algo¬ 
rithm  can  be  constructed:  On  each  timeslot,  observe  the  current  net¬ 
work  topology  state  S(t).  Given  that  S(t )  =  s,  randomly  choose  one  of 
the  L  +  1  control  options  Ps  with  probability  pls  (for  {1,...,L  +  1}). 
It  follows  that: 

E{n(t)\S(t)  =  s}  =  E{C(I(t),S(t))  \S(t)  =  s}  =  Gs.  (3.14) 

This  strategy  results  in  a  matrix  p{t)  consisting  of  individual  service 
rate  processes  m{t)  for  each  link  l.  Claim:  If  S(t)  evolves  according  to  a 
finite  state  irreducible  Markov  chain,  then  each  service  process  m(t)  is 
admissible  with  time  average  rate  Jii  =  Gp  A  formal  proof  of  the  claim 
is  given  in  [108].  Intuitively,  the  claim  holds  because  the  time  average 
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of  the  fi(t)  process  can  be  expressed  as  a  sum  over  the  steady  state 
topology  state  probabilities: 


lim  - 

t— XX)  t 


t~  1 


Y  ^(r) =  Y  7r*]S  (/*(*)  i5^) 

r— 0  sG«S 


s}  with  probability  1, 


and  hence  JL  =  G  by  (3.12)  and  (3.14).  Thus,  the  queue  for  each  link  l 
has  an  admissible  input  with  rate  A i  and  an  admissible  service  process 
with  time  average  rate  Jti  =  G[.  where  ~pi  is  strictly  larger  than  A i .  From 
the  sufficient  condition  of  Lemma  3.6  in  Section  3.1.1,  it  follows  that 
each  queue  of  the  network  is  stable,  proving  the  result.  □ 


The  above  stabilizing  algorithm  is  not  intended  as  a  practical  con¬ 
trol  strategy,  as  it  cannot  be  implemented  without  extensive  offline 
preparation.  Indeed,  the  input  rate  matrix  and  the  network  capacity 
region  would  need  to  be  known  in  advance,  which  requires  a-priori 
knowledge  of  the  topology  state  probabilities  tts  for  each  state  s  €  S. 
Further,  assuming  all  of  the  probabilities  could  be  accurately  estimated, 
the  network  controller  would  still  need  to  pre-compute  the  decomposi¬ 
tion  of  Gs  in  (3.13)  for  each  possible  topology  state.  As  the  number  of 
states  can  grow  geometrically  in  the  number  of  network  links,  a  direct 
attempt  to  implement  the  above  policy  would  be  very  difficult  even 
for  a  relatively  small  network.  However,  the  fact  that  the  above  policy 
exists  plays  a  crucial  role  in  the  analysis  of  a  more  practical  stabilizing 
strategy  presented  in  Chapter  4. 

Example  3.3.  The  capacity  of  ON-OFF  downlink.  Consider  a  sim¬ 
ple  example  of  a  two-queue  wireless  downlink  that  transmits  data  to 
two  downlink  users  1  and  2  over  two  different  channels.  Assume  that 
the  arrival  processes  are  independent  Bernoulli  processes  with  rates  Ai 
and  A2 ,  so  that  every  timeslot  a  single  packet  independently  arrives  to 
queue  i  with  probability  A,;,  and  no  packet  arrives  to  queue  i  other¬ 
wise  (for  i  £  {1,  2}).  All  packets  are  assumed  to  have  fixed  lengths,  so 
that  the  queue  backlog  is  measured  in  units  of  packets.  Let  U\{t)  and 
[/2(f)  respectively  represent  the  current  backlog  of  packets  waiting  for 
transmission  to  user  1  and  user  2,  respectively.  Channels  independently 
vary  between  ON  and  OFF  states  every  slot  according  to  independent 
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Bernoulli  processes  with  ON  probabilities  p±  and  p2 ■  Every  timeslot, 
a  controller  observes  the  channel  states  and  chooses  to  transmit  over 
either  channel  1  or  channel  2.  We  assume  that  a  single  packet  can  be 
transmitted  if  a  channel  is  ON,  and  no  packet  can  be  transmitted  when 
a  channel  is  OFF,  so  that  the  only  decision  is  which  channel  to  serve 
when  both  channels  are  ON.  In  this  case,  there  are  only  four  possible 
topology  states  S(t),  and  the  graph  family  T  is  given  as  the  following 
two  dimensional  set  of  rate  pairs  (<?i,  <72) ,  expressed  as  a  sum  of  sets  as 
in  (3.6): 

T  =  (1  -pi)(l  -p2){(0,0)}  +pi(l  -p2){(l,0)}+ 

P2(l  -  Pi){(0, 1)}  +  pip2Conv  {(1,  0),  (0, 1)}  .  (3.15) 

It  can  be  verified  that  the  resulting  capacity  region  of  this  system,  char¬ 
acterized  by  Corollary  3.10,  is  given  by  the  set  of  all  non-negative  rate 
vectors  (Ai,A2)  satisfying  the  following  three  inequalities  (in  addition 
to  Ai  >0,  A2  A  0): 

Ai  <  pi,  A2<p2,  Ai  +  A2  <  pi  +p2(l  -pi).  (3.16) 

This  is  a  polyhedral  region,  where  the  set  T  forms  the  dominant  face  of 
the  polyhedron  (See  Figure  4.1  in  the  next  section).  That  these  three 
inequalities  are  necessary  for  stability  is  quite  intuitive:  The  first  two 
inequalities  bound  the  individual  input  rates  A*  in  terms  of  the  max¬ 
imum  possible  average  transmission  rates  of  their  respective  queues, 
and  the  last  inequality  bounds  the  sum  input  rate  in  terms  of  the  aver¬ 
age  sum  transmission  rate.  This  simple  capacity  expression  arises  from 
the  special  ON / OFF  structure  of  the  system.  In  [148] ,  it  is  shown  that 
any  downlink  with  L  independent  ON/OFF  channels  has  a  capacity 
region  that  is  given  by  a  set  of  2L  —  1  inequalities:  Each  inequality  cor¬ 
responds  to  a  subset  of  channels  and  indicates  that  the  sum  input  rate 
into  this  subset  is  less  than  or  equal  to  the  probability  that  at  least 
one  channel  within  the  subset  is  ON.  A  similar  structure  for  the  capac¬ 
ity  region  holds  when  there  are  burstiness  constraints  on  the  channel 
state  processes  instead  of  the  Bernoulli  process  assumption  made  above 
[159], 
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In  this  chapter  we  develop  a  general  algorithm  for  stabilizing  networks 
without  requiring  knowledge  of  the  arrival  rates  or  topology  state  prob¬ 
abilities.  Unlike  the  offline  algorithm  considered  in  the  previous  chap¬ 
ter,  which  makes  randomized  resource  allocation  decisions  based  only 
on  the  observed  topology  state,  the  algorithm  in  this  chapter  is  an 
online  dynamic  algorithm  that  bases  decisions  both  on  the  observed 
topology  state  and  on  the  current  queue  backlogs.  We  begin  with  a 
motivating  example  that  illustrates  the  design  challenges. 


4.1  Scheduling  in  an  ON/OFF  Downlink 

Consider  the  ON/OFF  downlink  of  Example  3.3,  with  arrival  rates 
Ai,A2  and  independent  Bernoulli  channels  with  ON  probabilities  pi 
and  p-2,  and  assume  that  p\  <  P2-  Recall  that  the  capacity  region  of 
the  system  is  given  by  the  three  inequality  constraints  in  (3.16).  While 
the  controller  is  constrained  to  serving  only  a  single  queue  in  any  given 
timeslot,  the  fact  that  there  are  two  independent  channels  creates  a 
multi-user  diversity  gain ,  creating  a  larger  probability  that  at  least  one 
of  the  channels  is  ON  during  any  particular  timeslot.  This  gain  is  evi- 
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dent  in  the  inequalities  (3.16)  that  describe  the  capacity  region,  where 
it  is  clear  that  the  sum  output  rate  of  the  system  can  be  larger  than  the 
output  rate  of  any  single  queue  alone.  However,  this  diversity  gain  is 
mitigated  on  timeslots  in  which  one  of  the  queues  is  empty.  Therefore, 
even  for  this  simple  system,  scheduling  must  be  done  carefully  in  order 
to  ensure  stability. 

For  example,  assuming  the  arrival  rates  are  interior  to  the  capac¬ 
ity  region,  one  might  suspect  that  stability  can  be  achieved  simply 
by  serving  any  non-empty  ON  queue.  However,  in  the  case  when  the 
choice  is  to  serve  an  ON  queue  with  two  packets  versus  an  ON  queue 
with  twenty  packets,  serving  the  shorter  queue  can  potentially  create  a 
higher  probability  that  this  queue  is  empty  in  the  near  future,  leading 
to  a  loss  of  multi-user  diversity  and  creating  potential  instability.  This 
phenomenon  holds  also  for  multi-rate  systems:  Choosing  the  non-empty 
queue  with  the  largest  offered  transmission  rate  can  lead  to  instability 
and  sub-optimal  throughput,  even  though  this  policy  would  maximize 
throughput  in  the  special  case  when  all  queues  are  “infinitely  back- 
logged”  and  always  have  packets  to  send.  To  illustrate  this  point,  we 
compare  three  well  known  scheduling  algorithms  applied  to  the  special 
case  of  the  two  queue  downlink  with  ON/OFF  channels:  The  Borst 
algorithm  [24],  the  ‘proportionally  fair’  Max  Hi/ri  algorithm  [156],  and 
the  Max  Weight  Match  (MWM)  policy  [148]. 1 

The  Borst  Algorithm:  The  Borst  algorithm  chooses  to  serve  the  non¬ 
empty  channel  i  with  the  largest  /ij(i)/E  {/ij}  index,  where  is  the 
current  rate  offered  by  link  i  if  this  link  is  chosen  for  transmission,  and 
E{/b}  is  the  expected  value  of  this  rate  taken  over  its  steady  state 
distribution  (which  is  assumed  to  be  known  a-priori) .  This  algorithm  is 
shown  in  [24]  to  have  desirable  fairness  properties  for  wireless  networks 
with  an  “infinite”  number  of  channels,  where  each  incoming  packet  is 
destined  for  a  unique  user  with  its  own  channel.  Although  the  algo¬ 
rithm  was  not  designed  for  a  two-queue  downlink,  it  is  closely  related 
to  the  Max  /.q/r'i  policy  (to  be  described  below),  and  it  is  illuminating 
to  evaluate  its  performance  in  this  context.  Applied  to  the  two-queue 


1  The  MWM  policy  is  also  called  the  Longest  Connected  Queue  (LCQ)  policy  for  this  special 
case  of  ON/OFF  channels. 


42  Dynamic  Control  for  Network  Stability 


Fig.  4.1  The  downlink  capacity  region  A  and  the  stability  regions  of  the  Borst  policy  and  the 
Max  Hi/ri  policy.  Input  rates  (Ai,  A2)  are  pushed  toward  point  (0.5, 1.0),  and  the  simulated 
throughputs  under  the  Borst,  Max  fii/ri,  and  MWM  policies  are  illustrated. 


downlink,  the  Borst  algorithm  reduces  to  serving  the  non-empty  ON 
queue  with  the  largest  value  of  l/pt-  Because  p\  <  P2,  this  algorithm 
effectively  gives  packets  destined  for  channel  1  strict  priority  over  chan¬ 
nel  2  packets.  It  is  not  difficult  to  show  that,  for  this  simple  system, 
the  stability  region  of  the  Borst  policy  is  given  by  the  set  of  all  rate 
pairs  (Ai,  A2)  such  that  Ai  <  p\,  X2  <  P2  —  ^iP2  [119].  This  is  a  strict 
subset  of  the  capacity  region  (see  Fig.  4.1). 

The  Max  pi/ri  Algorithm:  Consider  now  the  related  policy  of  serv¬ 
ing  the  non-empty  queue  with  the  largest  value  of  p,i(t)/ri(t),  where 
ri(t)  is  the  empirical  throughput  achieved  over  channel  i.  This  differs 
from  the  Borst  algorithm  in  that  transmission  rates  are  weighted  by 
the  throughput  actually  delivered  rather  than  the  average  transmis¬ 
sion  rate  that  is  offered.  This  Max  pi/ri  policy  is  proposed  in  [156] 
and  shown  to  have  desirable  proportional  fairness  properties  when  all 
queues  of  the  downlink  are  infinitely  backlogged  [81]  [158].  To  evaluate 
its  performance  for  arbitrary  traffic  rates  (Ai,  A2),  suppose  the  running 
averages  r\(t)  and  7*2 (£)  are  accumulated  over  the  entire  timeline,  and 
suppose  the  system  is  stable  so  that  ?q(f)  and  7-2  (t)  converge  to  Ai  and 
A2.  It  follows  that  the  algorithm  eventually  reduces  to  giving  channel 
1  packets  strict  priority  if  Ai  <  A2,  and  giving  channel  2  packets  strict 
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priority  if  A2  <  Ai.  Thus,  if  Ai  <  A2  then  these  rates  must  also  satisfy 
the  inequalities  Ai  <  p\,  A2  <  P2  —  ^iP2  of  the  Borst  algorithm.  How¬ 
ever,  if  A2  <  Ai  then  the  rates  must  satisfy  the  inverted  inequalities 
A2  <  P2  and  Ai  <  pi  —  A2Pi-  Thus,  at  first  glance  it  seems  that  the 
stability  region  of  this  policy  is  a  subset  of  the  stability  region  of  the 
Borst  algorithm.  However,  its  stability  region  has  the  peculiar  property 
of  including  all  feasible  rate  pairs  (A,  A)  (see  Fig.  4.1). 

The  MWM  Algorithm:  The  MWM  algorithm  serves  the  queue  with 
the  largest  Ui(t)jjn(t)  index,  where  Ut(t)  is  the  current  backlog  in  queue 
i.  For  this  special  case  of  an  ON/OFF  downlink,  this  policy  reduces  to 
serving  the  longest  queue  with  an  ON  channel.  The  policy  is  shown  in 
[148]  to  stabilize  the  system  whenever  the  arrival  rates  are  interior  to 
the  capacity  region. 

In  Fig.  4.1  we  consider  the  case  when  pi  =  0.5, P2  =  0.6,  and  plot 
the  achieved  throughput  of  the  Borst,  Max  pi/rt,  and  MWM  poli¬ 
cies  when  the  rate  vector  (Ai,A2)  is  scaled  linearly  towards  the  vector 
(0.5, 1.0),  illustrated  by  the  ray  in  Fig.  4.1(a).  One  hundred  different 
rate  points  on  this  ray  were  considered  (including  example  points  o  - 
e),  and  simulations  were  performed  for  each  point  over  a  period  of  5 
million  timeslots.  Fig  4.1(a)  illustrates  the  resulting  throughput  of  the 
Borst  algorithm,  where  we  have  included  example  points  d!  and  e'  cor¬ 
responding  to  input  rate  points  d  and  e.  Note  that  the  Borst  algorithm 
always  results  in  throughput  that  is  strictly  interior  to  the  capacity 
region,  even  when  input  rates  are  outside  of  capacity.  Fig.  4.1(b)  illus¬ 
trates  performance  of  the  Max  pi/ri  and  MWM  policies.  Note  that  the 
MWM  policy  supports  all  (Ai,  A2)  traffic  when  this  rate  vector  is  within 
the  capacity  region.  However,  when  traffic  is  outside  of  the  capacity  re¬ 
gion  the  achieved  throughput  moves  along  the  boundary  in  the  wrong 
direction,  yielding  throughputs  that  are  increasingly  “unfair”  because 
it  favors  service  of  the  higher  traffic  rate  stream.  Like  the  Borst  policy, 
the  Max  /q/rj  policy  leads  to  instability  for  all  (stabilizable)  input  rates 
on  the  ray  segment  c-d,  and  yields  throughput  that  is  strictly  interior 
to  the  capacity  region  even  when  inputs  exceed  system  capacity  (com¬ 
pare  points  e  and  e').  However,  the  throughput  eventually  touches  the 
capacity  region  boundary  at  the  “proportionally  fair”  point  (0.4, 0.4) 
when  input  rates  are  sufficiently  far  outside  of  the  capacity  region. 
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This  example  illustrates  two  important  points.  First,  the  MWM  al¬ 
gorithm  provides  stability  whenever  possible,  while  other  reasonable 
algorithms  may  not.  This  first  point  is  a  special  case  of  a  general  sta¬ 
bility  result  for  networks  with  arbitrary  C(I(t),  S(t))  functions,  devel¬ 
oped  in  the  Section  4.4  via  a  theory  of  Lyapunov  stability.  Second,  the 
MWM  policy  does  not  necessarily  offer  fairness  in  cases  when  input 
rates  exceed  the  capacity  region.  This  issue  is  considered  in  Chapter 
5,  where  our  stabilizing  algorithms  are  complemented  with  an  optimal 
flow  control  technique  via  a  theory  of  Lyapunov  optimization. 

4.2  Network  Model 

We  consider  the  general  network  model  of  Chapter  2,  where  there  are 
N  nodes  and  K  commodities  (with  node  and  commodity  sets  J\f  and 
1C,  respectively).  The  network  is  characterized  by: 

•  A  topology  state  process  S(t)  that  evolves  according  to  an 
irreducible  Markov  chain  with  a  finite  state  space  S  and  time 
average  probabilities  irs  for  s  G  S. 

•  A  control  decision  variable  I(t)  (representing  resource  allo¬ 
cation)  with  a  potentially  topology  state-dependent  control 
space  Ls{t) . 

•  A  matrix  valued  transmission  rate  function  C(I(t),  S(t))  = 
(Cab(I(t),S(t))),  where  Cab(L(t),  S(t))  is  the  transmission 
rate  over  link  (a,  b)  under  the  control  action  I (t)  and  the 
topology  state  S(t)  (for  a,  b  €E  {1, . . . ,  IV}). 

Recall  that  the  Cab(I (t) ,  S (t))  functions  are  arbitrary,  and  are  only 
assumed  to  be  bounded.  Define  fi^  ( t )  as  the  routing  control  variables, 
representing  the  amount  of  commodity  c  data  delivered  over  link  (a,  b ) 
during  slot  t.  These  routing  variables  are  constrained  as  follows: 

K 

Y,  fJ$  (t)  <  Cab(I(t),S{t ))  for  all  (a,  b )  and  all  t,  (4.1) 

c=  1 

(*)  =  0  if  (a,  b)  i  Cc,  (4.2) 

where  Cc  is  the  set  of  all  links  that  are  allowed  to  transmit  commodity  c 
data.  Let  (t)  represent  the  process  of  exogenous  commodity  c  data 
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arriving  to  source  node  i  (for  i  £  J\[  and  c  £  1C).  We  assume  that  the 
flow  controllers  are  turned  off,  so  that  all  exogenous  arrivals  directly 
enter  the  network  layer  at  their  source  nodes.  Let  u\c\t)  represent  the 
backlog  of  commodity  c  data  currently  stored  in  node  i.  The  queueing 
dynamics  thus  satisfy: 

U-C\t  +  1)  <  max[C/f } \t)  -  Y  0]  +  A\c\t )  +  Y  Pah*)-  (4-3) 

b  a 

Each  process  A^'1  (t)  is  assumed  to  be  admissible  with  rate  less  than 
or  equal  to  \[L\  Define  A  =  (A,-*4)  as  the  matrix  of  arrival  rates.  We 
assume  that  the  input  rate  matrix  is  stabilizable,  and  in  particular  that 
is  within  the  relative  interior  of  the  capacity  region  A.  Recall  from 
Section  3.2  that  this  means  there  exists  a  value  e  >  0  such  that  A  +  e  £ 
A,  where: 

•  The  set  T>  contains  all  node-commodity  pairs  (i,  c )  for  which 
there  exist  network  queues  u\c\t). 

•  The  set  V  is  the  subset  of  V  consisting  of  all  node-commodity 
pairs  (i,  c )  for  which  it  is  possible  to  support  a  non-zero  rate 
Ajc)  (assuming  there  is  no  other  traffic). 

•  The  matrix  e  has  entries  e,-^  =  e  for  all  (i,c)  £  T>. 

Thus,  A  +  e  £  A  implies  that  it  is  possible  to  find  multi-commodity 
flows  to  support  the  augmented  traffic  rate  matrix  associated  with 
adding  a  new  stream  of  rate  e  to  each  of  the  source  queues  ( t )  (for 

( i ,  c)  £  V).  Data  from  any  new  stream  of  rate  e  is  simply  treated  as  if 
it  has  the  same  commodity  as  the  source  queue  it  enters.  To  simplify 
analysis,  we  assume  that  V  =  V.  This  is  equivalent  to  the  following 
assumption. 

“No  Trapping  Nodes'’  Assumption:  If  it  is  possible  to  send  commod¬ 
ity  c  data  to  a  particular  node  i  (so  that  a  commodity  c  queue  exists 
for  that  node),  then  it  is  possible  to  support  a  non-zero  communication 
rate  from  node  i  to  the  destination  of  commodity  c  (possibly  by  using 
multi-hop  paths). 

For  example,  a  “trapping  node”  might  be  a  node  with  no  outgoing 
links,  or  a  node  that  is  part  of  a  group  of  nodes  with  outgoing  links  that 
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only  connect  to  other  nodes  of  the  group.  Note  that  the  “no  trapping 
node”  assumption  holds  whenever  it  is  possible  to  deliver  data  back 
to  the  same  node  it  came  from.  While  this  assumption  is  not  required 
for  the  capacity  theorem  (Theorem  3.8),  it  shall  be  useful  in  analyzing 
the  performance  of  the  dynamic  control  policy  introduced  in  the  next 
subsection.  Indeed,  without  this  assumption,  a  general  dynamic  policy 
with  no  routing  constraints  might  inadvertently  send  data  to  a  trapping 
node  that  prevents  this  data  from  ever  reaching  its  destination. 

The  dynamic  policy  introduced  in  the  next  section  is  most  easily 
analyzed  under  the  no  trapping  node  assumption,  although  it  can  also 
be  applied  to  achieve  maximum  throughput  in  networks  without  this 
assumption.  In  practice,  if  a  particular  node  dies,  or  enough  of  its  out¬ 
going  links  die,  then  the  node  can  become  a  trapping  node.  In  this  case, 
although  all  of  the  data  contained  in  this  node  will  be  lost,  the  rate  of 
adding  more  data  to  this  node  will  approach  zero,  and  so  the  dynamic 
routing  policy  simply  finds  alternate  routes  for  all  future  data. 


4.2.1  Input  and  Output  Rate  Bounds 

To  analyze  network  performance,  it  is  useful  to  define  the  maximum 
transmission  rates  out  of  and  into  a  given  node  i  £  J\T  as  follows: 

N  N 

ftjmax,i=  ^  y  Cib(I,  ®)>  M max,i=  FFFT-  ^  ^  Cai(I ,  s). 

[sesjeis]  f-f  [seSjels]  “ 

Finite  values  of  the  above  constants  exist  because  the  C(/(f),  S'(t)) 
function  is  bounded.  To  simplify  network  analysis,  it  is  also  useful  to 
assume  that  the  total  exogenous  arrivals  to  any  node  i  €  J\f  are  deter¬ 
ministically  bounded  by  constants  ^4™aa:,  so  that  for  all  t  we  have:2 

j^4\t)<Arx 

ceJC 


2  The  deterministic  arrival  bound  is  not  necessary,  and  in  [108]  [122]  the  network  is  analyzed 
under  the  general  definition  of  admissible  inputs,  which  assumes  only  a  bound  on  the 
second  moment  of  exogenous  arrivals. 
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4.3  The  Stabilizing  Dynamic  Backpressure  Algorithm 

We  describe  below  an  algorithm  for  resource  allocation  and  routing 
which,  as  will  be  shown,  stabilizes  the  network  whenever  the  vector 
of  arrival  rates  lies  within  the  capacity  region  of  the  network.  The 
notion  of  controlling  the  system  to  maximize  its  stability  region  and 
the  following  algorithm  that  achieves  it  was  introduced  in  [142]  [147] 
and  generalized  further  in  [144]  [108]  [122], 

The  Dynamic  Backpressure  and  Resource  Allocation  Al¬ 
gorithm:  Every  timeslot  f,  the  network  controller  observes  the  queue 
backlog  matrix  U(t)  =  and  the  topology  state  variable  S(t) 

and  performs  the  following  actions  for  routing  and  resource  allocation. 

Resource  Allocation:  For  each  link  (a,  b),  define  the  optimal  com¬ 
modity  c*6(t)  as  the  commodity  that  maximizes  the  differential  backlog 
(ties  broken  arbitrarily): 

<&,(*)=  arg  max  fcc)(f)  -  t/fe(c)(f)l  , 

{c\(a,b)£Cc}  L  J 

and  define  W*b{t)  as  the  corresponding  optimal  weight: 

W^(t)Amax#(t))  (t)  -  U^{t)\t),  0],  (4.4) 

Choose  the  control  action  /(f)  that  solves  the  following  optimization: 

Maximize:  ^  W*b(t)Cab(I(t),  (4.5) 

ab 

Subject  to:  /(f)  £  Is(t)- 

Routing:  For  each  link  (a,  b)  such  that  W*b(t)  >  0,  offer  a  trans¬ 
mission  rate  of  ptab{t)  =  C ab(I (f) ,  S (t))  to  data  of  commodity  <&(*)■ 
Recall  that,  by  definition,  c*ab(t)  is  a  valid  commodity  to  send  over  link 
(a,  b )  (that  is,  (a,  b )  £  A:*6(t))-  If  there  is  not  enough  data  of  commod¬ 
ity  c*ab{t)  in  node  a  to  transmit  over  all  outgoing  links  requesting  this 
commodity,  idle  fill  bits  are  transmitted,  with  an  arbitrary  allocation 
of  actual  data  and  idle  fill  data  over  the  corresponding  outgoing  links. 

The  weights  W*b(t)  can  be  determined  at  each  node  provided  that 
nodes  are  aware  of  the  backlog  sizes  of  their  neighbors.  However,  the 
optimization  problem  (4.5)  that  must  be  solved  at  the  beginning  of 
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each  time  slot  requires  in  general  knowledge  of  the  whole  network  state. 
There  are  important  special  cases  where  this  optimization  problem  can 
be  solved  in  a  distributed  fashion  with  each  node  requiring  knowledge 
only  of  the  local  state  information  on  each  of  its  outgoing  channels.  This 
issue  will  be  described  in  Section  4.8.  The  resource  allocation  problem 
can  also  be  approximated  or  optimized  to  within  a  constant  factor  using 
the  schemes  that  will  be  described  in  Section  4.7. 

The  optimal  routing  commodities  c*6(t)  can  be  determined  provided 
that  nodes  are  aware  of  the  backlog  levels  of  their  neighbors.  Note  that 
this  routing  strategy  does  not  require  paths  to  be  specified  in  advance: 
Paths  are  chosen  dynamically  at  each  tinrestep  according  to  the  back¬ 
pressure  between  neighboring  nodes.  The  resulting  algorithm  assigns 
larger  transmission  rates  to  links  with  larger  differential  backlog,  and 
zero  transmission  rates  to  links  with  negative  differential  backlog. 

In  the  special  case  when  the  routing  constraint  sets  Cc  consist  of  all 
data  links,  then  the  above  policy  is  equivalent  to  the  Dynamic  Routing 
and  Power  Control  (DRPC)  policy  of  [122]  (where  the  /(f)  control 
variable  used  here  plays  the  role  of  the  P(t)  power  matrix  from  [122]). 
The  DRPC  policy  itself  is  a  generalization  of  the  original  backpressure 
algorithm  developed  for  multi-hop  packet  radio  networks  in  [147],  where 
stable  scheduling  algorithms  were  developed  using  maximum  weight 
activation  sets  with  link  weights  equal  to  differential  backlog. 

Consider  now  the  case  of  a  single-hop  network  where  a  commodity  c 
is  associated  with  each  link  (a,  b )  traffic.  As  mentioned  in  Section  2.2.1 
this  case  can  be  treated  by  simply  setting  Cc  =  {(a,  b)}  .  Taking  into 
account  that  the  traffic  backlog  at  the  destination  node  is  considered 
zero,  we  have  the  following  simplified  algorithm  in  this  case. 

The  Single-hop  Dynamic  Backpressure  and  Resource  Allo¬ 
cation  Algorithm:  Every  timeslot  f,  the  network  controller  observes 
the  queue  backlog  matrix  L7(t)  =  (  U%(t)  is  the  backlog  at  node 

a  of  traffic  destined  to  node  b)  and  the  topology  state  variable  S(t)  and 
performs  the  following  actions  for  routing  and  resource  allocation. 

Resource  Allocation:  Choose  the  control  action  I (t)  that  solves  the 
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following  optimization: 

Maximize:  Ub(t)Cab(I(t),  S(t)), 

ab 

Subject  to:  /(f)  6  ^s(t)  ■ 

Each  link  then  transmits  with  rate  /j,ab  (f)  =  Cab  (I  (f)  ,S(t)). 

We  close  this  section  by  describing  an  important  property  of  the 
proposed  algorithm;  in  fact  the  algorithm  is  designed  so  that  this  prop¬ 
erty  is  satisfied.  Consider  any  routing  control  variables  (t)  that  are 
admissible  (i.e. ,  satisfy  (4.1)  and  (4.2)).  Notice  that  by  the  definition 
of  the  Resource  Allocation  and  Routing  policy, 

EE'&’w  [c<c)(t)  -  (/‘ho] 

ab  c  ab  c 

<J2Wab(t)Cab(I(t),S(t)), 

ab 

where  the  final  inequality  follows  from  the  routing  constraints  (4.1) 
(4.2).  Moreover,  the  upper  bound  above  is  achievable  by  the  control 
policy  that  allocates  resources  to  maximize  the  weighted  sum  of  trans¬ 
mission  rates  ^2ab  W*b(t)Cab(I(t),  S(t))  subject  to  I(t )  £  %s(t)>  and 
then  chooses  control  variables: 


(c)/.n  =  f  Cab(I(t),S(t))  if  c  =  c*ab{t) 

ab  I  0  otherwise 


These  are  exactly  the  routing  control  variables  defined  by  the  Dynamic 
Backpressure  Algorithm.  Taking  into  account  the  following  simple  but 
important  identity 

ECT)  Ed?(‘)-Ed?(‘)  =EEdi(‘)[c<c)(t)-(/<c|(t)‘ . 

ic  Lb  a  J  ab  c 

we  conclude  from  the  above  the  following  basic  property: 

Basic  Property:  If  (t)  are  the  routing  control  variables  defined 
by  the  Dynamic  Backpressure  and  Resource  Allocation  Algorithm  and 
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ftf}{t)  those  of  any  other  feasible  algorithm,  then  for  any  time  t  >  0, 


E  A Cb  (*)  -  E  ^ab  (*) 


(c)  / 


L  b 


E^c)w 


< 


(c). 


L  6 


(4.7) 


In  the  next  section  we  provide  the  basic  tools  for  proving  stability 
via  Lyapunov  function  techniques,  which  are  then  used  in  Section  4.5 
for  proving  the  stability  of  the  Dynamic  Backpressure  Algorithm. 


4.4  Lyapunov  Stability 

One  of  the  most  important  mathematical  tools  for  proving  stability  of 
queueing  networks  and  for  developing  stabilizing  control  algorithms  is 
the  technique  of  Lyapunov  drift, .  The  idea  is  to  define  a  non-negative 
function,  called  a  Lyapunov  function,  as  a  scalar  measure  of  the  aggre¬ 
gate  congestion  of  all  queues  in  the  network.  Network  control  decisions 
are  then  evaluated  in  terms  of  how  they  affect  the  change  in  the  Lya¬ 
punov  function  from  one  slot  to  the  next. 

Specifically,  consider  a  network  with  L  queues,  and  let  U(t)  = 
. . . ,  U £,(i))  represent  the  vector  process  of  backlog  in  each  queue 
as  a  function  of  time.  We  define  the  following  quadratic  Lyapunov  func¬ 
tion  L(U ): 

L(U)  =  E 

i= 1 

Note  that  L(U(t))  =  0  if  and  only  if  all  network  queues  are  empty  at 
time  t,  and  that  L(U(t))  is  large  whenever  one  or  more  components  of 
U(t )  is  large. 

Assume  that  U(t)  evolves  according  to  some  probabilistic  law,  and 
that  the  initial  conditions  are  such  that  IE  {  [/*(())}  <  oo  for  all  queues 
L}. 


Lemma  4.1.  (Lyapunov  Stability)  If  there  exist  constants  B  >  0,  e  > 
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0,  such  that  for  all  timeslots  t  we  have: 


L 

E{L(U(t  +  l))-L(U(t))\U(t)}  <  B  -  e^Uitf), 

i= 1 


then  the  network  is  strongly  stable,  and  furthermore: 

1  .  r? 

lim  sup  -  IE  {[/-j(r)}  <  — . 

t — >oo  t  ^ 

T= 0 Z— 1 


(4.8) 


The  expression  on  the  left  hand  side  of  (4.8)  is  the  Lyapunov  drift, , 
representing  the  expected  change  in  the  Lyapunov  function  from  one 
slot  to  the  next.  With  slight  abuse  of  notation,3  we  shall  often  use 
A (U(t))  as  a  formal  representation  of  this  Lyapunov  drift: 

A(  U(t))4E  {L(  U(t  +  1  ))-L(  U(t))  |  U(t) }  . 


Note  that  if  the  condition  (4.8)  holds,  then  for  any  5  >  0,  the  Lyapunov 
drift  satisfies  A( U(t))  <  —5  whenever  l  ^*(0  A  (B  +  5)/e.  That  is, 
the  condition  of  the  Lemma  ensures  that  the  Lyapunov  drift  is  negative 
whenever  the  sum  of  queue  backlogs  is  sufficiently  large.  Intuitively,  this 
property  ensures  network  stability  because  whenever  the  queue  backlog 
vector  leaves  the  bounded  region 

(  I  L  'I 


U>  0 


Ui  <  (B  +  5)/e 


i— 1 


the  negative  drift  eventually  drives  it  back  to  this 


Proof.  (Lemma  4.1)  Assume  the  condition  (4.8)  holds  for  all  timeslots 
t.  Taking  expectations  of  (4.8)  (with  respect  to  the  distribution  for  the 
random  queue  backlog  U(t)  at  time  t)  we  have  by  the  law  of  iterated 
expectations: 

L 

E{L(U(t  +  l))}-E{L(U(t))}  <B-e^2E{Ui(t)}. 

i=  1 


3  Strictly  speaking,  the  Lyapunov  drift  should  be  expressed  as  A (U(t),t),  as  it  could  po¬ 
tentially  depend  on  the  timeslot  t  as  well  as  the  queue  backlog  values  U(t).  However,  to 
simplify  notation,  we  use  A (U(t))  as  a  formal  symbolic  representation  of  the  same  thing. 
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The  above  inequality  holds  for  all  timeslots  t  £  {0, 1,2,.. .}.  Summing 
the  inequality  over  timeslots  r  £  {0, . . . ,  M  —  1}  yields  a  telescoping 
series  on  the  left  hand  side,  resulting  in: 

M—l  L 

E  {L  (U(M))}  —  E  {L  (1/(0))}  <  BM-e  EE  E{^(r)}. 

T— 0  i— 1 

Dividing  the  above  inequality  by  M,  shifting  terms,  and  using  the  fact 
that  L(U(M))  >  0,  we  have: 


M—l  L 


sEEMWis 


T— 0  2—1 


B 

e 


E{L(U(0))} 

Me 


The  above  inequality  holds  for  all  positive  integers  M.  Taking  a  limsup 
as  M  oo  yields  the  result.  □ 


The  theory  of  Lyapunov  drift  has  a  long  history  in  the  field  of  dis¬ 
crete  stochastic  processes  and  Markov  chains  (see,  for  example,  [10] 
[106]).  The  first  application  of  the  theory  to  the  design  of  dynamic  con¬ 
trol  algorithms  for  radio  networks  appeared  in  [147],  where  a  general 
algorithm  was  developed  to  stabilize  a  multi-hop  packet  radio  network 
with  configurable  link  activation  sets.  The  concepts  of  maximum  weight 
matching  and  differential  backlog  scheduling,  developed  in  [147],  play 
important  roles  in  the  dynamic  control  strategies  we  present  in  later 
sections.  The  Lyapunov  drift  approach  has  been  successfully  used  to  op¬ 
timize  allocation  of  computer  resources  [22]  [21],  stabilize  packet  switch 
systems  [143]  [80]  [104]  [145]  [72]  [86],  satellite  and  wireless  systems 
[148]  [149]  [67]  [144]  [5]  [121]  [157],  and  ad-hoc  mobile  networks  [122], 
Recently,  a  simple  extension  of  Lyapunov  drift  theory  is  developed  in 
[108]  [109]  [119]  to  provide  both  stability  and  performance  optimization 
(addressed  in  more  detail  in  Chapters  5  and  6). 

Lyapunov  drift  theory  for  queueing  networks  is  traditionally  pre¬ 
sented  in  terms  of  Foster's  criterion  for  stability  (see,  for  example, 
[10]).  Roughly,  Foster’s  criterion  applies  to  queueing  processes  that 
evolve  according  to  ergodic  Markov  chains  with  countably  infinite  state 
spaces,  and  ensures  that  the  Markov  chain  has  a  well  defined  steady 
state  provided  that  some  mild  assumptions  hold,  and  provided  that 
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the  Lyapunov  drift  is  negative  whenever  the  queue  state  is  outside  of 
a  bounded  region  of  the  state  space.  The  form  of  the  Lyapunov  drift 
Lemma  (Lemma  4.1)  does  not  require  Foster’s  criterion,  and  is  adapted 
from  similar  statements  in  [86]  [67]  [121].  This  approach  to  Lyapunov 
stability  yields  a  simpler  stability  proof  as  well  as  an  upper  bound  on 
average  queue  occupancy.  If  the  conditions  of  Lemma  4.1  are  supple¬ 
mented  with  the  additional  Markovian  assumptions  of  Foster’s  crite¬ 
rion,  then  the  resulting  limsup  of  the  backlog  bound  in  Lemma  4.1  can 
be  replaced  with  a  regular  limit,  as  Foster’s  criterion  guarantees  the 
limit  exists. 

A  statement  similar  to  Lemma  4.1  can  also  be  made  concerning  T- 
slot  Lyapunov  drift,  which  is  useful  in  cases  when  network  stochastics 
require  more  than  one  timeslot  to  ensure  a  negative  drift: 


Lemma  4.2.  (T-slot  Lyapunov  drift)  If  there  is  a  positive  integer  T 
such  that  IE  {  U(t)}  <  oo  for  r  £  {0, . . . ,  T—  1},  and  if  there  are  positive 
values  B,  e  such  that  for  all  timeslots  to  we  have: 


L 

E  {L(U(to  +  T))  —  L(  U(t0 ))  |  U(t0) }  <  B  -  e  ]T  Ufa), 

i=  1 


then  the  network  is  strongly  stable,  and  the  average  congestion  satisfies: 


lim  sup  - 

t— >oo  t 


£X;E{^(r)}<f. 


The  proof  is  similar  to  the  proof  of  Lemma  4.1,  and  is  omitted  for 
brevity  (see  [108]  [122]  for  details). 

As  a  preliminary  demonstration  of  the  power  of  Lyapunov  drift 
theory,  we  use  the  T-slot  theorem  to  prove  the  sufficient  condition  for 
queue  stability  given  in  Lemma  3.6  of  Section  3.1.1.  The  next  simple 
lemma  will  be  useful. 


Lemma  4.3.  If  V,  U,  //,  A  are  nonnegative  real  numbers  and 


V  <  max  [U  —  n,  0]  +  A, 
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then 

V2  <  U2  +  ii2  +  A2  -  2U  {ft- A). 


Consider  a  single  queue  with  backlog  U{t)  and  with  arrival  and 
server  processes  A(t)  and 


Lemma  4.4.  If  A(t)  is  admissible  with  rate  less  than  or  equal  to  A, 
and  if  /i(f)  is  admissible  with  rate  greater  than  or  equal  to  ~p,  and  if 
A  <Ji,  then  the  queue  is  strongly  stable,  and  queue  backlog  satisfies: 

1  1  T(n2  4-  A2  ) 

limsup-  Ve{C/(t)}  <  x  maaJ,  (4.9) 

t— >  OO  t  ll  —  A 

T— U 

where  Umax  is  a  bound  such  that  fi(t)  <  Umax  for  all  t,  and  where  T 
is  the  smallest  integer  such  that  at  every  timeslot  t  and  (regardless  of 
past  history  of  the  system)  the  following  condition  holds: 


(  1  t+T-l  1  t+T—1 

B  r  E  ^ -f  E 

L  T=t  T=t 


A{r )  \H  (t)  >>(/!  —  A)/2.  (4.10) 


Further,  if  the  arrivals  A{t)  are  i.i.d.  every  slot  with  mean  IE  {Al(f)}  =  A, 
and  if  n(t)  service  rates  are  i.i.d.  every  slot  with  mean  E{/i(t)}  =  jJ, 
then: 

1  tA  (,,2  _L  a2  ) 

limsup  -Ve{[/(t)}  <  ^m0a.x_  .7°^. 

t— >oo  t  _  n  2(/i  A) 


(4.11) 


T  — 0 


The  parameter  T  used  in  the  above  theorem  can  be  viewed  as  the 
time  required  for  the  system  to  reach  “near  steady  state.”  Note  that 
the  general  bound  (4.9)  differs  by  a  factor  of  2 T  from  the  bound  (4.11) 
for  the  i.i.d.  case.  This  is  due  to  the  fact  that  non-i.i.d.  systems  may 
have  system  states  that  yield  low  transmission  rates  or  large  arrival 
bursts  for  many  timeslots  in  a  row.  While  the  i.i.d.  case  can  be  viewed 
as  a  special  case  when  T  =  1,  the  extra  factor  of  2  arises  because  i.i.d. 
systems  effectively  “reach  steady  state”  on  each  and  every  timeslot, 
so  that  the  left  hand  side  of  (4.10)  is  exactly  equal  to  ( jl  —  A)  for  all 
t.  Below  we  present  a  proof  of  Lemma  4.4  for  the  non-i.i.d.  case.  The 
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argument  below  is  the  only  T-slot  analysis  that  we  present  in  this  paper. 
In  later  sections  we  restrict  proofs  to  cases  that  involve  simpler  i.i.d. 
assumptions,  with  the  understanding  that  these  proofs  can  be  modified 
using  T-slot  analysis  to  yield  similar  results  for  non-i.i.d.  systems. 


Proof.  (Lemma  4.4)  The  unfinished  work  in  the  queue  T  slots  into 
the  future  can  be  bounded  in  terms  of  the  current  unfinished  work  as 
follows: 


U(t  +  T)  <  max 


t+T-l 

u(t)~  Y  +r)>° 

T=t 


t+T-l 

+  £  A(j). 


T=t 


The  above  expression  is  an  inequality  instead  of  an  equality  because 
new  arrivals  may  depart  before  the  T  slot  interval  is  finished.  From 
Lemma  4.3  we  have: 


/ t+T—l  \  2 

U\t  +  T)<  U\t )  +  T2n2max  +  f  Y  A^J 

-  t+T-l  t+T-l 

-2 TU(t)  -  Y  ^)~t  E  A (T) 

T=t  T=t 

Taking  conditional  expectations  with  respect  to  U  (t),  noting  that 

E{+r1)+r2)|C/(t)}  <  v/E{+n)2  +  +  }E{+r2)2  +  (t)}  <  A. 


2 

maxi 


and  using  the  definition  of  T  yields: 

E  {U2{t  +  T)  -  U2(t) \U{t ) }  <  T2^max  +  T2A2nax  -  2TU(t)(jI  -  A)/2. 

Applying  Lemma  4.2  to  the  above  inequality  (using  L(U )  =  U 2)  yields 
(4.9),  proving  the  result.  □ 


4.5  Lyapunov  Drift  for  Networks 

In  this  section  we  show  that  the  Dynamic  Backpressure  Algorithm  sta¬ 
bilizes  the  network,  using  Lyapunov  drift  techniques  described  in  Sec¬ 
tion  4.4.  Let  U(t)  represent  the  matrix  of  queue  backlogs,  and  define 
the  following  Lyapunov  function: 

l(u)  =  E(uic>)2- 

ic 
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The  above  sum  is  taken  over  all  (z,  c)  entries,  where  we  formally  define 
u[c)(t )  =  0  for  all  t  whenever  (z,  c )  ^  V.  To  calculate  the  drift  A(U(t)) 
we  apply  Lemma  4.3  to  the  queueing  equation  (4.3)  and  obtain, 

(c/f^  +  i))2  <  (V<c)(0)2+  +  (a(c)w +!>*(*)) 

Y  db  (*)  -  (*)  -  Y  Vai  (0  • 

.  b  a 

Summing  over  all  valid  entries  (z,  c)  and  using  the  fact  that  the  sum  of 
squares  of  non-negative  variables  is  less  than  or  equal  to  the  square  of 
the  sum,  it  is  not  difficult  to  show  that  the  above  inequality  implies: 


-  2 U[c\t) 


L(U(t  +  1))  -  L{U{t))  <  2 BN 


-2Yu\°\t) 

ic 


Y  ^ib  (*)  -  AiC)  (t)  -  Y  Vai  (*) 


where 

EIOCJm)1  +  WT*  +  (4.12) 

ieAf 

Taking  conditional  expectations  yields  the  following  bound  for  Lya¬ 
punov  drift: 


A(U(*))  <2BN  +  2Yut)m{A(f\t)\  U(t)} 


-2E<j  YUi°\t) 


Ydbit)  -Y^ai® 


(c)  / 


U(t)  .  (4.13) 


Using  the  Basic  Property  discussed  in  at  the  end  of  Section  4.3  we 
conclude  that  the  Backpressure  Algorithm  is  designed  to  minimize  at 
each  time  slot,  over  all  admissible  policies,  the  bound  in  the  right  hand 
side  of  (4.13).  This  is  the  key  property  on  which  the  proof  of  stability 
of  the  algorithm,  as  described  in  the  next  theorem,  is  based. 


Theorem  4.5.  (Backpressure  Algorithm  Performance)  If  there  exists 
a  value  e  >  0  such  that  A  +  e  6  A  (where  e  is  a  matrix  with  all  en¬ 
tries  (z,  c)  CD  equal  to  e,  and  all  other  entries  equal  to  zero),  then  the 
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above  dynamic  backpressure  algorithm  stabilizes  the  network.  Further¬ 
more,  in  the  special  case  when  the  arrival  processes  A(f)  are  i.i.d.  over 
timeslots  and  the  topology  state  process  is  i.i.d.  over  timeslots  (so  that 
E{A(f)}  =  A  and  Pr[S(t )  =  s]  =  tts  for  all  timeslots  t),  the  average 
congestion  satisfies: 
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where  emax  is  the  largest  value  of  e  such  that  A  +  e  6  A,  and  B  is 
defined  in  (4.12). 


The  theorem  is  proven  in  [122]  [108]  for  the  case  of  general  admis¬ 
sible  inputs  and  Markov  modulated  topology  state  processes,  where  a 
congestion  bound  is  also  derived.  As  in  the  case  of  single  hop  networks, 
the  congestion  bound  for  the  non-i.i.d.  case  is  roughly  a  factor  of  T 
larger  than  the  i.i.d.  congestion  bound  given  above,  where  T  is  the  du¬ 
ration  required  for  the  system  to  reach  “near  steady  state.”  Below  we 
prove  the  theorem  for  the  i.i.d.  case. 


Proof.  (Theorem  4.5)  For  simplicity,  we  assume  that  C7(T)  =  T.  Since 
arrivals  A(f)  are  i.i.d.  over  timeslots,  we  have  E  |  A^c\t)  \  U(t )  |  = 
for  all  (i,c).  Hence  we  can  rewrite  (4.13)  as: 


A  (U(f))  <  2 BN  +  2^  ulc)(t)\\c) 
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However,  recall  the  basic  inequality  (4.7),  which  states  that  the  Dy¬ 
namic  Backpressure  policy  minimizes  the  final  term  on  the  right  hand 
side  of  the  above  inequality  over  all  possible  alternative  policies  (t) . 
However,  because  A  +  e  6  A,  we  know  from  Corollary  3.9  of  Chap¬ 
ter  3  that  there  exists  a  stationary  randomized  algorithm  that  makes 
decisions  based  only  on  the  current  topology  state  5(f)  (and  hence  in¬ 
dependent  of  the  current  queue  backlog)  so  that  for  all  (i,c)  G  V  we 
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have: 

E  {  ^2  rib  (*)  “  rial  (<)  U(t)  }  =  e  +  A1C)  • 

l  b  a  ) 

Using  the  above  in  (4.14)  and  considering  (4.7),  we  conclude: 

A( U(t))  <  2BN  -  2e  EU’w- 

ic 

This  drift  inequality  is  in  the  exact  form  for  application  of  the  Lyapunov 
drift  lemma  (Lemma  4.1  of  Section  4.4),  proving  the  result.  □ 

4.6  Time  Varying  Arrival  Rates 

The  Dynamic  Backpressure  Algorithm  does  not  require  knowledge  of 
input  rates  or  topology  state  probabilities,  and  hence  it  easily  adapts 
to  time  varying  system  statistics  [108]  [120].  For  example,  suppose  the 
arrivals  A(i)  are  i.i.d.  with  expected  rate  vector  E{A(i)}  =  A*-1'  for 
some  duration  of  time  ti  <  t  <  t2,  but  that  user  demands  change  after 
time  t2,  so  that  E{A(i)}  =  A^2-*  for  t?  <  t  <  1 3.  After  time  £3,  the 
rates  might  change  again,  and  so  on,  so  that  rate  changes  occur  at 
arbitrary  times.  It  can  be  shown  that  the  system  is  strongly  stable  and 
has  average  congestion  bounded  by  a  uniform  constant,  provided  that 
there  is  a  positive  value  e  such  that  all  rate  matrices  are  within  a  given 
distance  e  of  the  capacity  region  boundary  [108]  [120]. 

4.7  Imperfect  Scheduling 

It  is  not  difficult  to  show  that  if  a  “sub-optimal”  control  decision  7(f) 
is  chosen  that  satisfies: 

J2w:b(t)Cab(I(t),S(t))  >7  max  ^  W*b(t)Cab(I(t),  S(t))  -  D, 

*  /eXs(t) 

for  some  constants  7,  D  such  that  0  <  7  <  1  and  0  <  D  <  00,  then  the 
network  is  also  stable,  provided  that  the  arrival  rates  are  interior  to  7A, 
which  is  a  7  scaled  version  of  the  capacity  region.  Thus,  if  the  controller 
is  off  from  the  optimum  by  no  more  than  an  additive  constant  D  (  i.e., 
7  =  1),  then  full  stability  is  still  possible  (although  the  resulting  con¬ 
gestion  increases  by  an  additive  constant  proportional  to  D ).  However, 
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if  the  controller  deviates  from  optimality  by  a  multiplicative  constant, 
the  achievable  throughput  region  may  be  a  subset  of  the  capacity  re¬ 
gion.  This  result  is  presented  in  [108],  and  similar  results  are  presented 
for  low  complexity  switch  scheduling  in  [134]  [120].  Related  “imperfect 
scheduling”  statements  are  developed  for  utility  optimization  in  [92]  us¬ 
ing  a  convex  optimization  theory  framework.  The  result  can  be  shown 
by  a  simple  modification  of  the  proof  of  Theorem  4.5.  Specifically,  the 
result  follows  by  replacing  (4.7)  with  the  following  inequality: 


'£  ut\t) 


£  £$  (*)  -  £  it b  (*) 


(c). 


E  uic)(b 


Edt’w-Edil*) 


—  D  < 

(c) 


L  b 


and  by  assuming  that  (A  +  e)  /q  G  A. 

This  simple  result  leads  to  two  significant  conclusions: 
First,  any  effort  to  allocate  resources  to  increase  the  value  of 
YliW*b(t)Ci(I(t),  S(t))  will  lead  to  improved  network  throughput, 
even  if  the  maximum  is  not  attained.  Second,  full  network  capacity 
can  be  achieved  by  using  queue  backlog  estimates ,  provided  that  the 
difference  between  the  estimate  and  the  exact  value  is  bounded  by  a 
constant  [108].  Therefore,  assuming  that  the  maximum  per-timeslot 
backlog  change  in  any  queue  is  bounded,  full  network  stability  can 
be  achieved  by  using  out  of  date  queue  backlog  information.  Queue 
updates  can  be  arbitrarily  infrequent  without  affecting  stability, 
although  the  average  congestion  may  increase  in  proportion  to  the 
duration  between  updates.  Related  work  in  the  area  of  imperfect 
scheduling  for  wireless  networks  is  developed  in  [92]  [163]  [29]. 


4.8  Distributed  Implementation 

As  mentioned  in  Section  4.3,  the  weights  W*b(t)  of  the  optimization 
problem  (4.5)  can  be  determined  by  node  a  provided  that  this  node  is 
aware  of  the  backlog  sizes  of  its  neighbors.  However,  the  optimization 
problem  itself  is  not  always  easily  amenable  to  a  distributed  solution, 
as  it  could  require  full  knowledge  of  the  topology  state  S(t)  and  full 
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coordination  of  all  network  nodes.  This  is  the  case  in  the  server  alloca¬ 
tion  problem  of  Example  2.3,  where  the  constraint  I(t)  £  I  can  only 
be  met  by  a  coordinated  decision,  and  in  the  power  allocation  problem 
of  Example  2.5  that  requires  knowledge  of  the  full  power  matrix. 

These  problems  can  be  avoided  if  the  network  is  designed  so  that  the 
link  rate  functions  Cab(I(t),  S(t))  depend  only  on  network  conditions 
and  control  input  decisions  that  are  local  to  link  (a,  6).  In  particular, 
suppose  that  the  topology  state  S(t),  the  link  control  input  /(f),  and 
the  control  space  I g(t)  can  be  decomposed  into  terms  associated  with 
G  independent  groups  of  nodes: 

S(t)  =  {s\t),  S\t),...,  SG(t)), 
m  =  {i\t),  /2(t),...,  iG(t)), 

Zs(t)  =  (t)  x  ^i2(*)  x  •  •  •  x  xsG(t)’ 

where  S9(t)  represents  the  local  components  of  the  topology  state  S(t) 
measured  at  nodes  within  group  g  (for  g  G  {1, . . . ,  G}).  Likewise,  I9{t ) 
represents  the  transmission  control  input  decisions  corresponding  to 
nodes  within  group  g,  and  satisfies  the  constraint  I9(t )  G  I9S  ^ .  Further 
suppose  that  the  link  transmission  rate  functions  for  each  outgoing  link 
(a,  b )  of  every  node  a  associated  with  a  particular  group  ga  can  be 
written  as  pure  functions  of  S9a(t )  and  I9a{t ): 

Cab(I(t),S(t))  =  Cab  (I9“(t),S9“(t)). 

In  this  case,  resource  allocation  decisions  associated  with  a  given 
group  g  can  be  made  independently  of  allocation  decisions  for  other 
groups.  An  example  network  where  such  a  decomposition  is  possible  is 
the  ad-hoc  mobile  network  of  Example  2.6,  where  groups  are  defined 
according  to  cell  partitions.  Similarly,  such  a  decomposition  is  possi¬ 
ble  when  communication  takes  place  over  wireline  data  links  that  do 
not  influence  other  channels,  or  when  all  wireless  network  nodes  trans¬ 
mit  over  orthogonal  frequency  bands  so  that  there  is  no  inter-channel 
interference. 

However,  networks  with  general  interference  properties  cannot  be 
decomposed  in  this  way.  One  approach  is  to  make  random  control  deci¬ 
sions  /(f),  and  then  to  have  the  flow  control  and  routing  layers  simply 
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react  to  the  resulting  transmission  rates  pab{t)  =  Cab(I(t),  S(t)).  This 
approach  is  considered  in  [57]  [108]  [122]  for  distributed  control  of  wire¬ 
less  ad-hoc  networks.  Another  approach  is  to  use  simplified  sub-optimal 
scheduling  to  achieve  results  within  a  constant  factor  of  optimality.  For 
example,  in  a  network  where  link  activation  sets  conform  to  matching 
constraints,  it  is  well  known  that  a  greedy  contention  based  schedul¬ 
ing  algorithm  achieves  within  a  factor  of  2  of  optimality.  Specifically, 
each  node  greedily  requests  to  transmit  over  its  maximum  weight  out¬ 
going  link,  and  conflicting  requests  are  resolved  by  granting  the  largest 
weight  contender  (breaking  ties  arbitrarily).  The  contention  scheme 
must  pass  through  several  iterations  before  reaching  a  point  when  no 
new  links  can  be  matched  (where  each  iteration  includes  at  least  one 
new  link).  Related  “factor  of  k”  results  apply  for  greedy  scheduling  in 
systems  where  each  link  has  at  most  k  other  interfering  links.  Greedy 
scheduling  strategies  of  this  type  are  considered  for  somewhat  differ¬ 
ent  control  algorithms  in  [92]  [163]  [29]  [164],  We  will  see  in  Chapter 
5  that  the  same  distributed  implementation  issues  arise  when  one  con¬ 
siders  resource  optimization  problems  and  fairness  issues  in  addition  to 
stability. 

4.9  Algorithm  Enhancements  and  Shortest  Path  Service 

Note  that  the  routing  constraint  sets  Cc  can  be  designed  to  ensure  that 
data  is  routed  over  links  that  make  progress  toward  the  appropriate 
destination.  However,  these  routing  restrictions  potentially  reduce  the 
network  capacity  region,  and  can  limit  adaptivity  when  link  or  node 
failures  necessitate  re-routing.  These  issues  can  be  avoided  if  all  sets  Cc 
are  equal  to  the  set  of  all  network  links.  While  this  leads  to  the  largest 
capacity  region  A,  it  can  also  lead  to  large  end-to-end  network  delay.  For 
example,  if  a  single  packet  is  injected  into  an  empty  network,  there  is  no 
backpressure  to  suggest  an  appropriate  path.  Hence,  the  packet  might 
take  a  random  walk  through  the  network,  or  might  take  a  periodic  walk 
that  never  leads  to  the  destination.  In  this  case,  although  the  network 
congestion  is  quite  low  (only  one  packet),  network  delay  can  be  infinite. 
Similarly,  in  cases  when  the  network  is  lightly  loaded,  the  end-to-end 
delay  can  be  large  even  though  the  congestion  bound  of  Theorem  4.5 
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is  satisfied. 

To  achieve  low  delay  while  still  avoiding  the  routing  restrictions 
associated  with  the  sets  jCc,  we  can  program  a  shortest  path  bias  into 
the  weights  of  the  dynamic  backpressure  algorithm.  This  leads  to  the 
following  “enhanced”  version  of  the  dynamic  backpressure  algorithm, 
defined  in  terms  of  constants  0\  >  0  and  Q\  >  0. 

Enhanced  Dynamic  Backpressure  Routing  Algorithm  (EDR):  For 
all  links  (a,  b),  find  the  commodity  c*ab(t)  such  that: 

cab(t)=  arg  cg  {OaHUaKt)  +  Qa} )  “  ^  (*)  +  Qft0)}  > 

and  define: 

W*b(t)=  max 

Control  decisions  are  then  made  as  before,  using  these  new  weights  and 
commodities  W*b(t )  and  c*ab{t)  as  a  replacement  for  the  originals. 

This  enhanced  strategy  is  developed  in  [108]  [122]  and  called  the 
“Enhanced  Dynamic  Routing  and  Power  Control  (EDRPC)”  strategy. 

Using  the  Lyapunov  function  L(U)  =  Ylic  ^  ^  >  it  is  n°t  difficult 

to  show  that  the  enhanced  algorithm  stabilizes  the  network  whenever 
the  original  algorithm  does. 

In  particular,  a  shortest  path  bias  can  be  programmed  into  the 
algorithm  by  setting  all  9)  ;  weights  to  1,  but  choosing  the  weights  Qi 
to  be  proportional  to  the  distance  (or  number  of  hops)  between  node 
i  and  the  destination  of  commodity  c  along  the  shortest  path  through 
the  network  (where  c/f'1  =  0  if  node  i  is  the  destination  of  commoditiy 
c).  These  distances  can  either  be  estimated  or  computed  by  running  a 
shortest  path  algorithm.  With  these  bias  values,  packets  are  inclined 
to  move  in  the  direction  of  their  shortest  paths,  providing  low  delay  in 
lightly  loaded  conditions  while  still  ensuring  stability  throughout  the 
entire  capacity  region. 

We  note  that  the  combined  weight  (t)  +  Q[r>  associated  with 
commodity  c  data  in  node  i  can  be  used  in  the  same  manner  as  a  rout¬ 
ing  table,  and  in  situations  where  the  network  nodes  do  not  change  their 
relative  locations,  the  unfinished  work  quantities  can  be  updated  each 


yCab) 


U^b\t)  +  Q{aab) 


Q'Cab> 

7b 


Ubab\t)  +  Q^ab 


Kb) 
b 


,0 


4.9.  Algorithm  Enhancements  and  Shortest  Path  Service  63 


timeslot  by  having  neighboring  nodes  transmit  their  backlog  changes 
over  a  low  bandwidth  control  channel.  As  each  link  transmits  only  a 
single  commodity  every  timeslot,  the  number  of  such  backlog  incre¬ 
ments  required  to  be  transmitted  over  the  control  channel  by  any  user 
is  on  the  order  of  neighboring  nodes. 

Below,  we  consider  the  special  case  of  a  simple  “multiple  source  sin¬ 
gle  sink"  static  sensor  network  consisting  of  100  sensor  nodes  randomly 
placed  in  a  10  x  10  square.  We  use  a  cell-partitioned  model  (similar  to 
the  one  considered  in  [109])  with  100  equal-sized  cells.  Each  node  can 
communicate  with  neighboring  nodes  in  the  same  or  adjacent  cells. 
Time  is  slotted  and  each  node  can  transmit  to  at  most  one  of  its  neigh¬ 
bors  in  a  time  slot,  though  a  node  can  receive  from  multiple  nodes. 
Each  sensor  node  has  independent  data  of  input  rate  A  to  deliver  to 
a  centrally  located  sink  node,  so  that  there  is  only  one  commodity  in 
the  network,  and  only  one  destination.  Nodes  are  assumed  to  have  a 
fixed  transmission  power  Pmax ■  The  data  rate  achievable  over  a  link 
is  then  taken  as  BWlog(  1  +  cuPmax)  where  a  is  the  attenuation  over 
that  link,  being  a  function  of  the  distance  between  the  nodes  and  B W 
is  the  bandwidth  of  the  link. 

We  consider  two  interference  scenarios,  one  in  which  a  node’s  trans¬ 
mission  doesn’t  cause  interference  at  other  nodes  and  the  other  where  a 
node’s  transmission  causes  interference  at  nodes  in  the  same  or  adjacent 
cells.  The  performance  of  a  distributed  implementation  of  DRPC  and 
EDRPC  is  compared  with  a  pure  shortest  path  based  routing  scheme 
in  this  setup.  The  left  plot  in  Fig.  4.2  shows  the  performance  of  these 
schemes  under  the  “no  interference"  model  with  asymmetric  bottleneck 
link  capacities.  The  right  plot  shows  the  performance  with  interfer¬ 
ence  and  symmetric  link  capacities.  It  can  be  seen  that  in  both  cases, 
DRPC/EDRPC  significantly  outperform  the  shortest  path  scheme  as 
input  rate  A  is  pushed  up.  At  low  data  rates,  backlog  based  decisions 
are  likely  to  lead  to  false  turns,  which  degrades  the  performance  of 
DRPC.  By  incorporating  path  lengths  into  the  backpressure  calcula¬ 
tion,  EDRPC  improves  upon  the  performance  of  DRPC  at  low  data 
rates  while  maintaining  the  advantages  of  backlog  aware  schemes  at 
high  data  rates. 
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Fig.  4.2  Simulation  results  comparing  backpressure  routing,  enhanced  backpressure  routing, 
and  pure  shortest  path  routing,  for  a  100  node  wireless  sensor  network  with  a  single  centrally 
located  destination. 


4.10  Multi-commodity  Flows  and  Convex  Duality 

The  Dynamic  Backpressure  Algorithm  stabilizes  the  network  and  offers 
average  delay  guarantees  whenever  the  input  rate  matrix  is  inside  the 
capacity  region  of  the  wireless  network.  Here  we  consider  a  related 
problem  of  computing  an  offline  multi-commodity  flow  given  a  known 
rate  matrix  Classical  multi-commodity  flow  problems  for  wired 

networks  can  be  reduced  to  linear  programs,  and  fast  approximation 
algorithms  are  developed  in  [85] .  A  distributed  algorithm  was  first  given 
in  [51],  and  pricing  and  game  theory  approaches  are  developed  in  [71] 

[70]- 

Here  we  consider  the  special  case  of  a  network  with  a  time  invariant 
topology  state,  and  formally  pose  our  network  stability  problem  as  a 
static  multi-commodity  flow  problem  (following  the  development  given 
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in  [108]  [122]).  We  show  that  a  classical  subgradient  search  method 
for  solving  the  problem  via  convex  duality  theory  corresponds  almost 
exactly  to  a  deterministic  network  simulation  of  the  Dynamic  Backpres¬ 
sure  Algorithm.  Notions  of  duality  are  also  used  in  [165]  [100]  [70]  [71] 
[101]  [66]  [84]  [31]  [79]  to  consider  static  network  optimization,  where 
in  many  cases  the  dual  variables  play  the  role  of  prices  charged  by 
the  network  to  multiple  users  competing  for  shared  network  resources 
in  order  to  maximize  their  own  utility.  Applications  of  static  duality 
theory  to  the  area  of  internet  congestion  control  are  developed  in  [97] . 

In  our  context,  the  dual  variables  correspond  to  queue  backlogs , 
rather  than  network  prices.  This  illustrates  a  relationship  between  sta¬ 
tic  optimization  and  the  Lyapunov  stability  theory,  suggesting  that  sta¬ 
tic  algorithms  can  be  modified  and  applied  in  dynamic  settings  while 
preserving  analytical  optimality.  This  observation  also  motivates  our 
study  of  stochastic  network  optimization  in  the  next  chapter. 

Consider  a  time  invariant  network  with  transmission  rate  function 
C(/(i))  for  I(t)  £  I  (note  that  there  is  no  notion  of  a  time  varying 
topology  state  here) .  Suppose  that  a  commodity  corresponds  to  a  des¬ 
tination,  so  that  all  commodity  c  data  is  destined  for  node  c.  Given  a 
particular  rate  matrix  (A  -c^),  the  problem  of  finding  a  multi-commodity 
flow  corresponds  to  the  following  convex  optimization  problem. 

Maximize  :  1 

Subject  to:  Xt\c)  +  ^  f[a?  <  ^  f^\i,  c  with  i  /  c, 

a  b 

{{fab}  AVab})  €  (4.15) 

where  Q  is  the  set  of  all  variables  ( j/ift1}  j  {lJ-ab}^  such  that: 

fab  >  0  for  all  a,  b,  c  €  {1, ... ,  N}, 

faa  =  fab  =  0  for  all  o,  b,  C  e  {1,  ...  ,  IV}, 

fab}  -  (hab)  for  some  (pab)  <E  C7{T}.  (4.16) 

The  maximization  function  “1”  is  used  as  an  artifice  to  pose  this  multi- 
commodity  flow  problem  in  the  framework  of  an  optimization  problem. 
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Note  that  the  set  Q  is  convex  and  compact  (it  inherits  convexity  and 
compactness  from  the  set  Cl(T)  consisting  of  all  link  transmission  rate 
matrices  (Gab)  entrywise  less  than  or  equal  to  some  element  of  T,  see 
[108]).  Moreover,  the  objective  function  “1”  and  all  inequality  con¬ 
straints  are  linear.  The  optimization  problem  is  therefore  convex  [19], 
and  has  a  dual  formulation,  where  the  optimal  solution  of  the  dual  prob¬ 
lem  exactly  corresponds  to  an  optimal  solution  of  the  original  ‘primal’ 
problem  (4.15).  To  form  the  dual  problem,  we  introduce  non-negative 
Lagrange  multipliers  { ujc^ }  for  each  of  the  inequality  constraints  in 
(4.15),  and  define  the  dual  function: 


L 


max 


1  + 


E'Wz/f-  £/<? 

i^c  \  b  cl 


The  dual  problem  to  (4.15)  is: 

Minimize:  L  (W1)> 

Subject  to:  >0  for  alH,  c  €  {1, . . . ,  IV}. 


(4.17) 


The  dual  problem  is  always  convex,  and  the  minimizing  solution  can  be 
obtained  using  classical  subgradient  search  methods  (where  the  func¬ 
tion  -l({c//c)})  is  maximized).  Consider  a  fixed  stepsize  method 
with  stepsize  T  =  1.  The  basic  subgradient  search  routine  starts  with 
an  initial  set  of  values  U^c\ 0)  for  the  Lagrange  multipliers,  and  upon 
each  iteration  t  =  {1,2,...}  these  values  are  updated  by  computing  a 
subgradient  r]  for  one  time  unit,  and,  if  necessary,  projecting  the  result 
back  onto  the  set  of  non-negative  values  [19]: 


u\c\t  + 1) 


max 


U?Xt)  +  Tjf>,  0 


(c) 


(4.18) 


However,  it  is  shown  in  [19]  that  a  particular  subgradient  of 


—L 


(K’}) is; 


2=1  -E4w+E/:m+a 


*(c) 


(4.19) 
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where  the  |  1  variables  are  solutions  to  the  maximization  in  (4.17) 

with  I/E  =  U-C\t).  Using  (4.19)  in  (4.18)  for  all  i  /  c,  we  find: 


U-C\t  +  1)  =  max 
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ic)(t)  - 


\  ^ 
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Kc) 


\  ^ 


(4.20) 


From  the  above  equation,  it  is  apparent  that  the  Lagrange  multipliers 
|  llj0'1  (t)  |  play  the  role  of  unfinished  work  in  a  multi-node  queueing 

system  with  input  rates  A jc,  where  U^c\t)  represents  the  amount  of 
commodity  c  bits  in  node  i.  In  this  way,  the  f*j.)C'1  values  can  be  viewed 
as  the  transmission  rates  allocated  to  commodity  c  traffic  on  link  (a,  b). 
Equation  (4.20)  thus  states  that  the  unfinished  work  at  time  t  +  1  is 
equal  to  the  unfinished  work  at  time  t  plus  the  net  influx  of  bits  into 
node  i.  Thus,  the  operation  of  projecting  the  Lagrangian  variables  onto 
the  positive  orthant  acts  exactly  as  an  implementation  of  the  standard 
queueing  equation. 

It  is  illuminating  to  calculate  the  optimal  /*E  values  by  perform¬ 
ing  the  maximization  in  (4.17).  To  this  end,  we  need  to  maximize 

'Zi^cUiC\t)  (^btib  ~  Ea/i?)  subject  to  the  constraints  of  (4.16). 
However,  as  in  Section  4.5,  we  can  switch  the  sum  to  find: 


EU’fe/iUE/i? 

i^c  \  b  cl 


\  "  Ac) 

I ab 


abc 


uF  -  ulc) 


Remarkably,  from  the  right  hand  side  above,  it  is  apparent  that  the 
optimal  values  f*^  are  identical  to  the  resulting  link  rates  (P)  that 
would  be  computed  if  the  DRPC  algorithm  were  used  to  calculate  rout¬ 
ing  and  power  allocation  decisions  in  a  network  problem  with  unfinished 
work  levels  U^c\t).  It  follows  that  the  DRPC  algorithm  can  be  viewed 
as  a  dynamic  implementation  of  a  subgradient  search  method  for  com¬ 
puting  the  solution  to  an  optimization  problem  using  convex  duality. 
This  suggests  a  deeper  relationship  between  stochastic  network  con¬ 
trol  algorithms  and  subgradient  search  methods.  Further,  it  suggests 
an  approach  to  stochastic  network  optimization.  Indeed,  note  that  the 
optimization  problem  (4.15),  which  maximizes  the  function  “1,”  can 
be  adjusted  to  maximize  some  other  performance  criteria,  which  may 
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be  of  interest  in  the  corresponding  dynamic  network  control  problem. 
One  approach  is  to  use  the  theory  of  stochastic  approximation  and  sto¬ 
chastic  subgradients  (see  for  example  [68]  [44]),  which  has  recently  been 
applied  in  [83]  to  a  downlink  scheduling  problem  with  infinitely  back- 
logged  sources.  In  the  next  chapter,  we  develop  a  method  that  uses  a 
novel  extension  of  Lyapunov  drift  theory  to  allow  stability  and  perfor¬ 
mance  optimization  simultaneously  [108]  [109]  [119],  yielding  explicit 
tradeoffs  in  utility  optimization  and  average  delay. 


5 


Networking  Outside  of  the  Capacity  Region:  Utility 
Optimization  and  Fairness 


Up  to  this  point  we  have  focused  attention  only  on  the  problem  of 
controlling  a  network  to  achieve  stability.  In  this  chapter  we  begin  our 
treatment  of  stochastic  network  optimization,  where  the  goal  is  to  stabi¬ 
lize  the  network  while  additionally  optimizing  some  performance  metric 
and/or  satisfying  some  additional  constraints.  Specifically,  this  chapter 
investigates  the  situation  when  the  exogenous  arrival  rates  are  outside 
of  the  network  capacity  region  A.  In  this  case,  the  network  cannot  be 
stabilized  without  a  transport  layer  flow  control  mechanism  to  limit 
the  amount  of  data  that  is  admitted.  The  goal  is  to  design  a  cross¬ 
layer  strategy  for  flow  control,  routing,  and  resource  allocation  that 
provides  stability  while  achieving  optimal  network  fairness.  Here,  we 
measure  fairness  in  terms  of  a  general  utility  function  of  the  long  term 
admission  rates  of  each  session. 

The  solution  to  this  flow  control  problem  involves  three  new  con¬ 
cepts.  The  first  is  a  simple  extension  of  Lyapunov  drift  theory  that 
enables  stability  and  performance  optimization  to  be  treated  simulta¬ 
neously  [108]  [109]  [119].  The  second  is  the  introduction  of  auxiliary 
variables  that  are  used  to  hold  additional  state  information  useful  for 
optimizing  functions  of  time  averages  [119]  [111]  [115].  The  third  is 
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the  technique  of  using  virtual  cost  queues  to  transform  performance 
constraints  into  queueing  stability  problems  [109]  [119]  [136].  These 
techniques  are  instrumental  in  the  design  of  utility-optimal  flow  con¬ 
trollers,  and  shall  also  be  used  in  Chapter  6  to  address  more  general 
problems  of  network  optimization,  including  minimum  energy  routing 
and  cost  constrained  scheduling. 

5.1  The  Flow  Control  Model  and  Fairness  Objective 

Consider  the  general  multi-hop  network  model  introduced  in  Chapter 
2,  where  the  network  has  N  nodes,  I\  commodities,  a  topology  state 
process  5(f),  and  a  link  transmission  rate  function  C(/(f),  S(t ))  (where 
/(f)  6  Tgu\).  Recall  that  a  general  network  control  algorithm  must 
choose  the  resource  allocation  and  routing  decision  variables  as  follows: 


•  Resource  (Rate)  Allocation:  Observe  the  current  topology 
state  5(f)  and  choose  a  transmission  control  /(f)  £  Rs(t)  to 
yield  link  transmission  rates  /x(f)  =  C(/(f),  5(f)). 

•  Routing /Scheduling:  For  each  link  (a,  b)  and  each  commodity 
c,  choose  (f)  to  satisfy  the  following  constraints: 


^2  Rab  (t)  <  nab{t),  (5.1) 

C 

Rab  it)  =  0  if  (a,  b)  i  Cc,  (5.2) 

(where  Cc  is  the  set  of  all  network  links  that  are  acceptable 
for  commodity  c  data  to  traverse). 


Newly  arriving  data  does  not  immediately  enter  the  network  layer. 
Rather,  it  first  enters  a  transport  layer  storage  reservoir.  Specifically, 
new  data  of  commodity  c  that  arrives  to  source  node  n  is  first  placed  in 
a  transport  layer  reservoir  (n,  c).  A  control  valve  determines  the  amount 
of  data  Rn\t)  released  from  this  reservoir  on  each  timeslot.  This  Rn\t ) 
process  acts  as  the  exogenous  arrival  process  to  the  network  layer  queue 
Un\t).  As  discussed  in  Section  2.2,  the  following  inequality  holds  for 
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the  network  queues. 


U^\t+l)  <  max 


Let  Ln  '1  ( t )  represent  the  current  backlog  in  the  transport  layer  reservoir 
(n,  c)  at  time  t.  The  flow  control  decision  variables  R^  (t)  are  chosen 
every  timeslot  according  to  the  following  restriction: 


•  Flow  Control  (Type  1):  Choose  Rn\t )  such  that: 

R$  ( t )  <  L$  (■ t )  +  A$  ( t )  for  all  (n,  c)  and  all  f , 
R$(t)  <  R™ax  for  all  n  and  all  t, 

C 

where  the  constants  R™ax  are  chosen  to  be  positive  and  suit¬ 
ably  large,  to  be  made  precise  in  the  development  of  our 
CLC1  flow  control  algorithm  (introduced  in  the  next  sec¬ 
tion).  The  first  constraint  above  ensures  that  admitted  data 
is  less  than  or  equal  to  the  actual  data  available,  and  the 
second  is  important  for  limiting  the  burstiness  of  the  admit¬ 
ted  arrivals.  We  label  the  above  flow  control  constraints  as 
“Type  1”  to  distinguish  them  from  the  following  alternative 
constraint  specifications: 

•  Flow  Control  (Type  2):  Choose  Rn\t )  such  that: 

Rn\t )  <  L^\t)  +  A^\t)  for  all  (n,  c)  and  all  t, 

R ^  ( t )  <  for  all  n  and  all  t, 

"(c) 

where  Rn '  are  suitably  large  positive  constants,  to  be  made 
precise  in  our  CLC2  algorithm  in  Section  5.4.2.  The  Type  2 
constraints  are  simpler  but  are  also  less  restrictive  and  gen¬ 
erally  lead  to  a  larger  bound  on  average  network  congestion. 


5.1.1  The  Fairness  Objective 

Let  A  =  denote  the  arrival  rate  matrix  of  the  exogenous  arrival 

streams  ( An\t )).  This  rate  matrix  is  arbitrary,  and  in  particular  the 
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rates  can  either  be  inside  or  outside  of  the  capacity  region  A.  Flow 
control  decisions  about  what  data  to  admit  are  crucial  in  the  case  when 
the  input  rates  exceed  network  capacity,  and  it  is  important  to  establish 
a  quantitative  measure  of  network  fairness.  To  this  end,  we  define  a  set 
of  utility  functions  gn\r),  representing  the  “satisfaction”  received  by 
sending  commodity  c  data  from  node  n  to  the  destination  node  of  this 
commodity  at  a  time  average  rate  of  r  bits/slot.  The  utility  functions 
are  assumed  to  be  non-decreasing  and  concave.  Such  utility  functions 
are  a  conventional  means  of  measuring  network  fairness.  For  example, 
concave  utilities  are  used  to  evaluate  fairness  for  wireline  networks  in 
[70]  [71]  [97]  [107],  for  static  wireless  networks  in  [84]  [16]  [66]  [31] 
[92],  and  for  stochastic  wireless  networks  in  [108]  [119]  [45]  [136]  [83]. 
Furthermore,  different  choices  of  the  gn\r )  functions  lead  to  different 
fairness  properties  [140]  [70]. 

For  our  problem,  the  goal  is  to  support  a  fraction  of  the  traffic 
demand  matrix  A  to  achieve  a  long  term  throughput  matrix  r  = 
that  maximizes  the  sum  of  user  utilities.  The  optimal  sum  utility  is 
thus  defined  by  the  following  optimization  problem: 


Maximize:  (r£c)) 

n,c 

(5.4) 

Subject  to:  re  A, 

(5.5) 

0  <  v <  A^  for  all  (n,  c). 

(5.6) 

Inequality  (5.5)  is  the  stability  constraint  and  ensures  that  the  long 
term  admitted  rates  are  stabilizable  by  the  network.  Inequality  (5.6)  is 
the  demand  constraint  that  ensures  the  admission  rate  of  session  (n,  c) 
is  no  more  than  the  incoming  traffic  rate  of  this  session. 

Because  the  functions  gn(r )  are  non-decreasing,  it  is  clear  that  if 
As  A,  then  the  above  optimization  is  solved  by  the  matrix  r*  =  A  (so 
that  r =  AnC'  for  all  (n,  c)).  If  A  ^  A,  then  the  solution  r*  will  lie 
somewhere  on  the  capacity  region  boundary.  The  above  optimization 
could  in  principle  be  solved  if  the  arrival  rates  (\n>)  and  the  capacity 
region  A  were  known  in  advance,  and  all  users  could  coordinate  by 
sending  data  according  to  the  optimal  solution.  However,  the  capacity 
region  depends  on  the  topology  state  probabilities,  which  are  unknown 
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to  the  network  controllers  and  to  the  individual  users.  Furthermore, 
the  individual  users  do  not  know  the  data  rates  or  utility  functions 
of  the  other  users.  In  this  chapter,  we  develop  a  practical  dynamic 
control  strategy  that  yields  a  resulting  matrix  of  throughputs  r  that  is 
arbitrarily  close  to  the  optimal  solution  of  (5.4)-(5.6).  The  distance  to 
the  optimal  solution  is  shown  to  decrease  like  1/F,  where  V  is  a  control 
parameter  affecting  a  tradeoff  in  average  delay  for  data  that  is  served 
by  the  network. 

5.1.2  Capacity  Region  Geometry 

Recall  that  D  is  the  set  of  all  (n,  c )  pairs  that  represent  valid  network 
layer  queues  Un'1  ( t )  (so  that  it  is  possible  for  commodity  c  data  to  be 
present  at  node  n).  The  integer  D  represents  the  total  number  of  these 
queues,  and  defines  the  effective  dimension  of  the  network.  We  assume 
throughout  that  Un\t)= 0  ,  Rn\t)= 0  for  all  t  whenever  (n,  c)  ^  V , 
and  that  g„\r)= 0,  An^O  whenever  (n,c)  ^  V. 

The  capacity  region  A  can  be  shown  to  be  compact  and  con¬ 
vex  with  D  effective  dimensions  [108].  It  shall  be  useful  to  define 
the  parameter  nsym  to  be  the  largest  time  average  admission  rate 
that  is  simultaneously  supportable  by  all  sessions  (n,  c)  £  V,  so  that 
(fisym^n^)  £  A  (where  In'*  is  an  indicator  function  that  is  equal  to 
1  whenever  (n,  c)  £  V ,  and  zero  else).  Geometrically,  the  value  nsym 
represents  the  edge  size  of  the  largest  H-dimensional  hypercube  that 
can  be  fit  into  the  capacity  region  A,  and  is  a  value  that  unexpectedly 
arises  in  our  analysis.  We  assume  throughout  that  gsym  >  0. 

In  the  next  section,  we  present  a  solution  to  the  fairness  problem 
(5.4)-(5.5)  in  the  special  case  when  all  active  sessions  are  infinitely 
backlogged  (so  that  the  demand  constraint  (5.6)  is  removed).  A  modified 
algorithm  that  uses  auxiliary  variables  and  flow  state  queues  is  then 
presented  in  Section  5.4  to  solve  the  general  problem  (5.4)-(5.6). 

5.2  Dynamic  Control  for  Infinite  Demand 

Here  we  develop  a  practical  control  algorithm  that  stabilizes  the  net¬ 
work  and  ensures  that  utility  is  arbitrarily  close  to  optimal,  with  a 
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corresponding  tradeoff  in  network  delay.  We  define  an  active  session 
(n,  c)  to  be  a  source-commodity  pair  such  that  (n,  c)  €  V  and  gffl  (r) 
is  not  identically  zero.  To  highlight  the  fundamental  issues  of  routing, 
resource  allocation,  and  flow  control,  in  this  section  we  assume  that 
all  active  sessions  (n,  c)  have  infinite  backlog  in  their  corresponding 
reservoirs,  so  that  flow  variables  Rn\t )  can  be  chosen  without  first  es¬ 
tablishing  that  this  much  data  is  available  for  admission.  Flow  control 
is  imperative  in  this  infinite  backlog  scenario,  and  the  resulting  problem 
is  simpler  as  it  does  not  involve  the  demand  constraint  (5.6). 

The  following  control  strategy,  developed  in  [108]  [119],  is  decou¬ 
pled  into  separate  algorithms  for  resource  allocation,  routing,  and  flow 
control. 

Cross-Layer  Control  Algorithm  1  (CLC1)  [108]  [119]: 

•  Flow  Control  —  (algorithm  FLOW1)  Every  timeslot,  the  flow 
controller  at  each  node  n  observes  the  current  level  of  queue 
backlogs  Un\t)  for  each  commodity  c  6  {1, . . . ,  K }.  It  then 
sets  R^\t)  =  Tn  \  where  the  values  are  solutions  to  the 
following  optimization: 

K 

Maximize  :  ^  [v  gft  (ric))  -  r$u£\t)  ,  (5.7) 

c=  1 

I< 

Subject  to:  (ric))  >  0,  ?ic)  <  R™ax,  (5.8) 

C— 1 

where  V  >  0  is  a  chosen  constant  that  effects  the  perfor¬ 
mance  of  the  algorithm. 

•  Routing  and  Scheduling  —  Each  node  n  observes  the  backlog 

in  all  neighboring  nodes  j  to  which  it  is  connected  by  a 
valid  outgoing  link  (n,j).  Let  W^(t)  =  Un\t )  —  UjC\t) 
represent  the  differential  backlog  of  commodity  c  data,  and 
define  max[c|ie£c]  {w^)(t),oJ.  Let  c*nj(t)  represent 

the  maximizing  commodity.  Data  of  commodity  c*n]  (t)  is 
selected  for  (potential)  routing  over  link  (n.  j)  whenever 

>  0. 
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•  Resource  Allocation  —  The  current  topology  state  S(t)  is 
observed,  and  a  transmission  decision  I(t)  £  2 gm  is  se- 
lected  by  maximizing  Xm  j  where  = 

Cnj(I(t),  S(t)).  The  resulting  transmission  rate  of  nnj(t )  is 
offered  to  commodity  c*  -(f)  data  on  link  (■ n,j ).  If  any  node 
does  not  have  enough  bits  of  a  particular  commodity  to  send 
over  all  outgoing  links  requesting  that  commodity,  null  bits 
are  delivered. 

The  flow  control  policy  (5.7)-(5.8)  uses  a  parameter  V  that  deter¬ 
mines  the  extent  to  which  utility  optimization  is  emphasized.  Indeed,  if 
V  is  large  relative  to  the  current  backlog  in  the  source  queues,  then  the 
admitted  rates  Rn\t )  will  be  large,  increasing  the  time  average  util¬ 
ity  while  consequently  increasing  congestion.  This  effect  is  mitigated 
as  backlog  grows  at  the  source  queues  and  flow  control  decisions  be¬ 
come  more  conservative.  Note  that  the  routing  and  resource  allocation 
strategies  of  CLC1  are  identical  to  the  Dynamic  Backpressure  strat¬ 
egy  developed  for  network  stability  in  Section  4.3.  Issues  of  distributed 
implementation  can  be  dealt  with  using  the  methods  of  Section  4.8. 

The  flow  control  algorithm  of  CLC1  is  decentralized,  where  the  con¬ 
trol  valves  for  each  node  n  require  knowledge  only  of  the  queue  back¬ 
logs  in  node  n.  Note  that  CLC1  uses  “Type  1”  flow  control  constraints. 
The  resulting  problem  (5.7)-(5.8)  involves  maximizing  a  sum  of  con¬ 
cave  functions  subject  to  a  simplex  constraint,  and  can  easily  be  solved 
using  standard  derivative  matching  techniques  [17].  It  is  useful  to  note 
that  this  flow  control  strategy  can  be  replaced  by  a  strategy  that  uses 
“Type  2”  flow  control  constraints,  and  thus  maximizes  (5.7)  over  the 
less  restrictive  constraint  set  0  <  rn  <  R™ax .  This  constraint  set  is 
simpler  because  it  allows  each  flow  control  decision  variable  Rn\t)  to 
be  determined  by  finding  the  maximum  of  a  concave  function  of  one 
variable  over  a  given  interval,  a  problem  in  which  closed  form  solutions 
are  often  readily  available.  However,  this  simplicity  comes  at  the  cost 
of  a  potential  increase  in  average  network  congestion  and  delay,  due  to 
the  fact  that  the  maximum  admission  burst  into  node  n  during  a  single 
slot  would  be  KR™ax,  rather  than  R™ax  (see  delay  analysis  in  Section 
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5.3).  The  simplified  constraint  set  is  identical  to  (5.8)  in  the  special 
case  when  there  is  only  a  single  active  session  (n,  c)  at  any  given  node 
n,  considered  in  examples  in  Sections  5. 2. 2-5. 2. 4. 

We  also  note  that  the  nature  of  the  CLC1  flow  control  algorithm 
assumes  that  data  can  be  admitted  as  “fractional  packets.”  This  is 
because  the  resulting  Rn\t )  values  might  not  be  integers  or  integer 
multiples  of  a  given  packet  length.  This  problem  arises  whenever  the 
utility  functions  gn\r )  are  non-linear.  The  problem  can  be  mitigated 
when  R™ax  is  large  in  comparison  to  the  packet  granularity,  or  can  be 
overcome  by  appending  an  additional  stage  to  the  flow  control  reser¬ 
voir  that  only  sends  actual  packets  into  the  network  when  the  accumu¬ 
lated  “admitted  but  undelivered”  data  exceeds  the  packet  length.  The 
problem  is  avoided  entirely  in  the  modified  algorithm  CLC2  that  uses 
auxiliary  variables  to  handle  non-linear  effects  (Section  5.4.2). 

5.2.1  Algorithm  Performance 

To  analyze  the  performance  of  the  above  CLC1  algorithm,  we  define 
the  maximum  transmission  rates  out  of  and  into  a  given  node  n  as 
follows: 

N  N 

Vmax,n=  .  max  ^  °nb^  ’  S )’  Vmax,n=  .max  Can(I ,  S ). 

[Sjels]  “  [SJ^ls]  “ 

Assume  that  the  flow  control  constants  R™ax  are  positive  and 
satisfy  R™ax  >  Rmaxn  f°r  all  n-  Assume  utilities  gn\r)  are 
non-negative,  non-decreasing,  continuous,  and  concave,  and  define 
Gmax=  ^2crn^ <R^axNn\  £ n,c9n]  (ric)).  Define  the  constant 

B  as  follows: 

1  N 

B=2JV  £  [  w 11  +  rZ«*,„)2  +  .  (5.9) 

n=l 


Theorem  5.1.  If  channel  states  are  i.i.d.  over  timeslots  and  all  active 
reservoirs  have  infinite  backlog,  then  for  any  flow  parameter  V  >  0  the 
CLC1  algorithm  stabilizes  the  network  and  yields  the  following  time 
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average  congestion  and  utility  bounds:1 


Y,  £ 


NB  +  VG„ 


l^sym 


(5.10) 


iimmf  Y  9n\rnC(t))  >  Y  9nC)  (<(C))  “  C5-11) 

n,c  n,c 

where  r*  =  ( r,* is  the  optimal  solution  of  (5.4)  subject  to  constraint 
(5.5),  and  where: 


^*#Alimsup^  E 1  { ^ (T) } 

n,c  t— 0  L  ti,c 

rnc(t)Al^E{r(lc)(r)}  . 


T— 0 


(5.12) 


The  above  result  holds  for  all  V  >  0.  Thus,  the  value  of  V  can  be 
chosen  so  that  BN /V  is  arbitrarily  small,  resulting  in  achieved  utility 
that  is  arbitrarily  close  to  optimal.  This  performance  comes  at  the  cost 
of  a  (potential)  linear  increase  in  network  congestion  with  the  para¬ 
meter  V .  By  Little’s  theorem,  average  queue  backlog  is  proportional 
to  average  bit  delay,  and  hence  performance  can  be  pushed  towards 
optimality  with  a  corresponding  tradeoff  in  end-to-end  network  delay. 
We  note  that  in  the  special  case  when  the  input  rate  matrix  is  inside 
the  capacity  region,  a  tighter  bound  than  (5.10)  can  be  computed,  one 
that  does  not  depend  on  the  V  parameter  and  has  a  form  similar  to 
the  bound  derived  in  Theorem  4.5  in  Section  4.5  for  the  differential 
backlog  policy  without  flow  control. 

A  result  similar  to  the  above  theorem  holds  if  the  “Type  1”  flow  con¬ 
trol  constraint  ]TC  4?i-f ^  (t)  <  R™ax  in  (5.8)  is  replaced  with  the  simpler 
and  less  restrictive  “Type  2”  constraint  Rn\t)  <  R™ax  (for  all  (n,  c)). 
However,  the  constant  B  in  the  performance  bounds  (5.10)  (5.11)  would 


1  Using  T-slot  Lyapunov  analysis,  the  CLC1  algorithm  can  be  shown  to  yield  similar  per¬ 
formance  for  non-i.i.d.  systems  (with  an  increased  constant  B).  The  same  is  true  for  the 
other  performance  optimal  algorithms  presented  in  this  work. 
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then  need  to  be  replaced  with  a  constant  B  that  is  larger  than  B  roughly 
by  a  factor  equal  to  the  largest  number  of  distinct  commodities  that 
are  sourced  at  any  single  node. 

The  proof  of  Theorem  5.1  uses  a  Lyapunov  technique  that  allows 
stability  and  performance  optimization  to  be  treated  simultaneously, 
and  is  provided  in  Section  5.3.  Below  we  present  a  simple  corollary  that 
is  useful  in  characterizing  the  performance  of  CLC1  under  suboptimal 
resource  allocation  strategies: 

Corollary  5.2.  If  the  resource  allocation  policy  of  CLC1  is  replaced 
with  any  (potentially  randomized)  policy  that  yields  a  transmission 
rate  function  fi(t)  =  C(I(t),  S(t))  that  satisfies  the  following  for  all 
slots  t: 

^W:bm{Rab(t)\U(t)}>1(  max  £  W*ab(t)Cab{I ,  S(t))  )  -C, 
ab  S(t)  ab  J 

for  some  fixed  constants  7  and  C  such  that  0  <  7  <  1  and  C  >  0,  then 


jj(c)  ^  C  +  NB  +  V Gmax 
n,c  RsymTi 

(5.13) 

lim  inf  > 

t—>  OO  J 

71.. r. 

n..r. 

(5.14) 

where  (f*c)  is  the  optimal  solution  to  the  following  optimization: 

Maximize:  ^  3n }  (^ ) 

(5.15) 

n,c 


Subject  to:  (r^)  £  jA, 

0  <  ric)  <  A. 


That  is,  allocating  resources  to  come  within  a  factor  7  of  the  optimal 
solution  of  the  CLC1  resource  allocation  yields  a  utility  that  is  close  to 
the  optimal  utility  with  respect  to  a  7  scaled  version  of  the  capacity 
region.  The  above  corollary  is  related  to  the  approximate  Lyapunov 
scheduling  results  presented  in  Section  4.7.  A  closely  related  “imperfect 
scheduling”  result  is  developed  from  a  convex  programming  perspective 
in  [92],  Below  we  present  the  implications  of  these  results. 
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5.2.2  Maximum  Throughput  and  the  Threshold  Rule 

Suppose  utilities  are  linear,  so  that  (r)  =  ancr  for  some  non-negative 
weights  anc ■  The  resulting  objective  is  to  maximize  the  weighted  sum 
of  throughput,  and  the  resulting  FL0W1  algorithm  of  CLC1  has  a 
simple  threshold  form,  where  some  commodities  receive  as  much  of  the 
R™ax  delivery  rate  as  possible,  while  others  receive  none.  In  the  special 
case  where  the  user  at  node  n  desires  communication  with  a  single 
destination  node  cn  (so  that  gn\r )  =  0  for  all  c  /  cn),  the  flow  control 
algorithm  (5.7)  reduces  to  maximizing  V anCnr  —  Unn\t)r  subject  to 
0  <  r  <  R'nax,  and  the  solution  is  the  following  threshold  rule: 


Rnc„  ( t ) 


R™ax  if  Unn\t)  <Va: 

0  otherwise 


The  qualitative  structure  of  this  flow  control  rule  is  intuitive:  When 
backlog  in  the  source  queue  is  large,  we  should  refrain  from  sending  new 
data.  The  simple  threshold  form  is  similar  to  the  threshold  scheduling 
rule  developed  in  [159]  for  server  scheduling  in  a  downlink  with  N 
ON/OFF  channels  and  burstiness  constraints  on  the  channel  states 
and  packet  arrivals.  Specifically,  assuming  an  <  an+ \  1  <  n  <  N  —  1, 
the  policy  developed  in  [159]  assigns  at  time  t  to  channel  queue  n  an 
index 


In  {U  (tj)  =  min  {U  ( t ) ,  nT}  , 


where  T  is  a  given  threshold  parameter,  and  transmits  a  packet  from  the 
ON  queue  with  the  highest  index.  It  is  shown  that  for  T  large  enough 
this  policy  maximizes  the  weighted  sum  of  channel  throughputs.  In  the 
special  case  of  this  downlink  scenario  (a  commodity  is  identified  with  a 
channel),  the  resource  allocation  layer  of  the  CLC1  policy  would  always 
serve  the  longest  ON  queue,  and  the  flow  control  layer  would  only  allow 
new  packets  of  type  i  into  queue  i  if  the  current  backlog  in  this  queue  is 
below  V  (and  so  backlog  in  queue  i  would  never  grow  beyond  V +R™ax). 
In  comparison,  we  see  that  the  V  and  T  parameters  play  similar  roles. 
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5.2.3  Proportional  Fairness  and  the  l/U  Rule 

Consider  now  utility  functions  of  the  form  gn\r)  =  log  ^1  +  (for 

some  constant  j3  >  0).  It  is  shown  in  [70]  that  maximizing  a  sum  of 
such  utilities  over  any  convex  set  A  leads  to  proportional  fairness.2  In 
the  special  case  when  there  is  only  one  destination  cn  for  each  user  n, 
the  flow  control  algorithm  reduces  to  maximizing  V  log(l  +  fir)  —  Un  " ■* r 
subject  to  0  <  r  <  Rffax ,  which  leads  to  the  following  ‘l/U’  flow  control 
function: 


Rncn  if) 


min 


max 


V 

uiCn\t) 


T>max 
5  -^n 


Here  we  see  that  the  flow  control  valve  restricts  flow  according  to  a 
continuous  function  of  the  backlog  level  at  the  source  queue,  being  less 
conservative  in  its  admission  decisions  when  backlog  is  low  and  more 
conservative  when  backlog  is  high. 


5.2.4  Mechanism  Design  and  Network  Pricing 

The  flow  control  policy  (5.7)  has  a  simple  interpretation  in  terms  of 
network  pricing.  Specifically,  consider  a  scenario  where  the  gfi  (r)  func¬ 
tions  are  measured  in  units  of  dollars,  representing  the  amount  the  user 
at  source  node  n  is  willing  to  pay  for  rate  r  service  to  destination  c. 
The  social  optimum  operating  point  is  defined  as  the  point  that 

maximizes  the  sum  of  utilities  J2nc9n^  (r«^)  subject  to  6  A. 

For  simplicity,  we  again  assume  that  there  is  a  single  user  at  node  n 
that  desires  to  send  a  single  commodity  cn.  Every  timeslot,  each  user 
n  determines  the  amount  of  data  Tnn\t )  it  desires  to  send  based  on 


2  Strictly  speaking,  the  proportionally  fair  allocation  seeks  to  maximize  c  ’  )•  lead- 

fopt  _r(c)  /  \ 

ing  to  c  nc_optn  >  0  for  any  other  operating  point  ( r ^ ')  E  A.  We  use  non-negative 

’  rnc 

utilities  log(l  +  fir)  and  thereby  obtain  a  proportionally  fair  allocation  with  respect  to 

—ovt 

the  quantity  rnc  +  1  /fi,  leading  to  c  — 777  ^  0-  This  can  be  used  to  approximate 
proportionally  fair  scheduling  for  large  fi.  Alternatively,  it  can  be  used  with  fi  =  1,  yielding 
a  utility  function  log(l  +  r)  which  is  different  from  proportionally  fair  utility  but  still  has 
many  desirable  fairness  properties. 
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a  per-unit  price  PRICEnCn(t )  charged  by  the  network.  The  transac¬ 
tion  between  user  and  network  takes  place  in  a  distributed  fashion  at 
each  node  n.  We  assume  all  users  are  ‘greedy’  and  send  data  every 
timeslot  by  maximizing  total  utility  minus  total  cost,  subject  to  an 
R™ax  constraint  imposed  by  the  network.  That  is,  each  user  n  selects 
Rnn\t )  =  rnn\  where  the  riCn')  values  solve: 

Maximize  :  g ^  -  PRICEnCn(t)r^n\  (5.16) 

Subject  to:  0  <  <  R™ax. 

Consider  now  the  following  dynamic  pricing  strategy  used  at  each 
network  node  n: 

PRICEnc{t)  =  ;  dollars/bit. 

We  note  that  this  pricing  strategy  is  independent  of  the  particular 
9n^  (r)  functions,  and  so  the  network  does  not  require  knowledge  of  the 
user  utilities.  Using  this  pricing  strategy  in  (5.16),  it  follows  that  users 
naturally  send  according  to  processes  R^  ( t )  that  exactly  correspond  to 
the  FLOW1  algorithm  (5.7),  and  hence  the  performance  bounds  (5.10) 
and  (5.11)  are  satisfied. 

5.3  Performance  Analysis 

To  prove  Theorem  5.1,  we  first  introduce  a  simple  result  from  [108]  [109] 
[119]  that  extends  the  Lyapunov  stability  results  presented  in  Lemma 

4.1  of  Section  4.4  to  include  performance  optimization.  In  particular, 
the  technique  allows  queueing  stability  and  performance  optimization 
to  be  treated  using  a  single  drift  analysis. 

5.3.1  Lyapunov  Optimization 

To  begin,  consider  any  discrete  time  queueing  system  with  vector  back¬ 
log  process  U(t)  =  . . .  ,Uv(t))  that  evolves  according  to  some 

probability  law,  and  let  R(t.)  =  (Ri(t), . . . ,  Rxit))  represent  any  as¬ 
sociated  vector  control  process  that  influences  system  dynamics  (for 
some  integers  N  and  K ).  Let  L(TJ)  be  any  non-negative  function  of  U. 
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Recall  that  the  one-step  Lyapunov  drift  A(U(t))  is  defined  as  follows: 

A(  U(t)) 4E  {L(  U(t  +  1  ))-L(  U(t ))  |  U(t) }  , 

where  the  conditional  expectation  is  taken  with  respect  to  the  random 
one-step  queueing  dynamics  given  the  current  backlog  U(t). 


Lemma  5.3.  (Lyapunov  Drift)  Let  E{L(£/(0))}  <  oo.  If  there  exist 
scalar  random  processes  x(t )  and  y(t )  such  that  for  every  timeslot  t 
and  for  all  possible  values  of  17(f),  the  Lyapunov  drift  satisfies: 

A (U(t))  <  E  {y(t)  |  U(t) }  -  E  {x(t)  \  U(t ) }  ,  (5.17) 


then: 


t- 1 


t- 1 


lim  sup  -  E  {x(t)}  <  lim  sup  -  E  E{y(r)} , 

t— >00  t  r,  t— >00  t 

t=U  r=0 

^  t- 1  j  t-1 

lim  inf  -  E  {x(t)}  <  lim  inf  -  E  {y(r)}  . 

t— >00  f  <  f— >r*n  t  z—j 


T— 0 


t— >00  f 


T= 0 


Proof.  Taking  expectations  of  (5.17)  with  respect  to  the  distribution 
of  U(t)  and  using  the  law  of  iterated  expectations  yields: 

E{L(U(t  +  1))}  -E{L(l/(f))}  <  E{y(t)}  -E{x(f)}. 


The  above  inequality  holds  for  all  t.  Summing  over  t  G  {0 , ,M  —  1} 
yields: 

M—l  M  —  1 

E{L(U(M))}-E{L(U(0))}<  ^E{y(r)}-  £>{®(r)}. 

T  — 0  T  — 0 

Shifting  terms,  dividing  by  Af,  and  using  non-negativity  of  L(TJ )  yields: 


1 

M 


M- 1 


^  E{x(r)}  < 

T  — 0 


1 

M 


M-l 

2  E{y(T)}  + 


T  — 0 


E{L(U(0))} 

M 


The  result  follows  by  taking  limits  as  M  — >  oo. 


□ 
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Suppose  now  that  the  goal  is  to  stabilize  the  U(t)  process  while  min¬ 
imizing  a  concave  function  g{-)  of  the  time  average  of  the  R(t )  process. 
Specifically,  we  define  the  following  vector  of  time  average  expectations 
over  t  slots: 

r(t)A|^E{R(i)}.  (5.18) 

T— 0 

Let  g(r)  be  any  scalar  valued,  concave  utility  function  of  a  K  dimen¬ 
sional  variable  r,  and  let  g*  represent  a  desired  “target”  utility  value. 


Theorem  5.4.  (Lyapunov  Optimization)  If  there  are  positive  con¬ 
stants  V,  e,  B  such  that  for  all  timeslots  t  and  all  unfinished  work  ma¬ 
trices  U(t),  the  Lyapunov  drift  satisfies: 


N 

A(  17(f))  -  VE{g{R(t))\U(t)}  <  B  -  e  £  U&)  -  Vg\ 

i=  1 

then  time  average  utility  and  congestion  satisfies: 
t- 1  N 


lim  sup  -  EE  E{Li(r)}  < 


i— xx) 


t 


T— 0  2—1 


B  +  V{g-g*) 


B 


lim  inf  g(r(t))  >  g*  -  — , 
t— >oo  V 

where  r(f)  is  defined  in  (5.18),  and  g  is  defined: 


t- 1 


g =  lim  sup  -  ^  IE  {g(R(T))}  . 


t — >oo 


T  — 0 


(5.19) 


(5.20) 

(5.21) 


This  theorem  is  most  useful  when  the  quantity  (g  —  g*)  can  be 
bounded  by  a  constant.  For  example,  if  0  <  g(R(t))  <  GmaX  for  all 
t  (for  some  constant  Gmax),  then  (g  —  g*)  <  Gmax.  It  follows  that  if 
the  parameter  V  can  be  chosen  as  desired,  then  the  lower  bound  on 
achieved  utility  can  be  pushed  arbitrarily  close  to  the  target  utility  g*, 
with  a  corresponding  increase  in  queue  congestion  that  is  linear  in  V. 
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Proof.  (Theorem  5.4)  Define: 

x(t)=eY^ui(t)  +  Vg* , 

i 

y(t)=B  +  Vg{R(t)). 

The  drift  condition  (5.19)  thus  implies: 


A(  U(t))  <  E  {y(t)  |  U(t) }  -  E  {*(t)  |  U(t) }  . 


(5.22) 


Using  the  lirnsup  result  of  Lemma  5.3  yields: 


lim  sup  -  Y^  E  j  e  Y"'  Ut(t)  +  V g*  \  < 

t~*°°  1  T=0  {  i  J 

1 

lirnsup  -  ^  E{i?  +  Vg(R(t))}  . 


t — >oo 


T— 0 


The  right  hand  side  of  the  above  inequality  is  equal  to  B  +  Vg.  Re¬ 
arranging  terms  yields  (5.20).  Likewise,  using  (5.22)  with  the  lim  inf 
result  of  Lemma  5.3  yields: 


t- 1  t-i 

lim  inf  -  Y^E{x(f)}  <  lim  inf  -  Y^E{i?  +  Vg(R(t ))} 

t— KX)  t  ^ J  f  A — J 


T  — 0 


t — >oo  t 


T  — 0 


t~  1 


B  +  V  lim  inf  -  {#(!£(£))}  . 

t — >oo  t  ^ J 


T= 0 


Noting  that  x(t)  >Vg*  yields: 


t- 1 


Vg*  <  R  +  Uliminf-  YE{5(R(t))}. 

t— >  OO  t  ^ ' 


T  — 0 


Dividing  by  U  and  rearranging  terms  yields: 


1 


t-i 


B/V  <  liminf  -  Y^E  {g(R(t))} 

t—> OO  £  -a — J 


T  — 0 


/f 

<  lim  inf  g  (  -  E  {R(r)} 


T  — 0 


(5.23) 

(5.24) 
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where  (5.24)  follows  from  Jensen’s  inequality  together  with  concavity 
of  g(-).  It  follows  that  liminf^oo  g(r(t))  >  g*  —  B/V,  proving  the 
theorem.  □ 

It  is  useful  to  note  that  a  similar  result  can  be  shown  for  minimizing 
a  convex  cost  function  h(r)  by  defining  g(r)  =  — h(r )  and  reversing 
inequalities  where  appropriate.  Further,  the  proof  uses  concavity  of  g(r) 
only  in  the  inequality  (5.24).  Thus,  without  the  concavity  assumption, 
the  congestion  bound  (5.20)  still  holds,  while  the  utility  bound  (5.24) 
is  replaced  by  (5.23). 

5.3.2  Computing  the  Drift 

To  apply  the  Lyapunov  Optimization  Theorem  (Theorem  5.4)  to  the 
design  and  analysis  of  our  CLC1  control  policy,  we  define  the  util¬ 
ity  function  g(r)=  Yn  cgP  •  This  utility  function  is  used  to 

evaluate  the  utility  associated  with  the  flow  control  decision  vari¬ 
ables  R(t)  =  ^ Rn\t )^.  Further,  we  define  the  Lyapunov  function 

L(U)  =  \  Yn  c  (uP'j  (the  factor  1/2  is  used  for  notational  convenience 
later).  The  Lyapunov  Optimization  Theorem  suggests  that  a  good  con¬ 
trol  strategy  is  to  greedily  minimize  the  following  drift  metric  every 
timeslot: 

A(U(t))  -Vj2®{g{n}  (4C)(0)  I  U(t)  }  ■  (5.25) 

n,c 

This  is  indeed  the  principle  behind  the  CLC1  control  algorithm. 

To  begin,  recall  that  the  Lyapunov  drift  A( U(t))  for  any  control 
policy  can  be  computed  using  methods  in  Section  4.5,  and  we  have: 

A(  U{t))  <NB~Y,  (t)E  I  ^  ^2 (*) 

n,c  L  b 

(5.26) 

a  J 

where  B  is  defined  in  (5.9).  The  expectations  above  are  taken  with 
respect  to  the  distribution  of  the  random  topology  state  S(t)  at  time 
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t,  and  with  respect  to  the  (potentially  randomized)  choice  of  control 
decision  variables. 

Now  define  the  flow  function  \I/(  77(f))  and  the  network  function 
4>(77(f))  as  follows: 


*(U(t))fl^E{VgicX^(t))  -  uV(t)RU(t) \U(t)}  ,  (5.27) 

HU(t)) A  /$(*)  -  I  U(t)  }  •  (5.28) 


n,c  ^  b  a  ) 

Subtracting  the  utility  component  V^2n  CE  \  U(t)  | 

from  both  sides  of  (5.26)  yields: 


A (U(t))-V^s{g^  (i#)(f))  |U(f)}  < 

n,c 


1V5  -  $(77(7)) (77(f)).  (5.29) 


The  design  strategy  for  CLC1  is  now  apparent:  Given  a  particular  17(f) 
matrix  at  time  t,  the  CLC1  policy  is  designed  to  greedily  minimize  the 
right  hand  side  of  (5.29)  over  all  possible  routing,  resource  allocation, 
and  flow  control  options.  Indeed,  it  is  clear  that  the  flow  control  strategy 
(5.7)  maximizes  \I/(  77(f))  over  all  feasible  choices  of  the  Rn\t)  values 
(compare  (5.7)  and  (5.27)).  That  the  routing  and  resource  allocation 
policy  of  CLC1  maximizes  <h( 77(f))  has  already  been  seen  in  Section 
4.3,  where  the  differential  backlog  policy  was  presented.  Our  analysis 
proceeds  by  finding  a  stationary  control  policy  for  choosing  the  decision 
variables  that  does  not  depend  on  the  queue  backlog,  and  plugging  the 
resulting  4>(U"(t))  and  T(t7(f))  functions  associated  with  such  a  policy 
into  the  right  hand  side  of  (5.29). 


5.3.3  A  Near-Optimal  Operating  Point 

To  design  a  suitable  stationary  control  policy,  it  is  important  to  first 
consider  a  near- optimal  solution  to  the  optimization  problem  (5.4)- 
(5.6).  Specifically,  for  any  e  >  0,  we  define  the  set  At  as  follows: 

Ae=  |  (r$  +  el^)  G  A,  >  0  for  all  (n,  c)  G  V  j  ,  (5.30) 

where  In'1  is  equal  to  1  whenever  (n, c)  £  T> ,  and  zero  else.  Thus,  the 
set  Ae  can  be  viewed  as  the  resulting  set  of  rate  matrices  within  the 
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network  capacity  region  when  an  “e-layer”  of  the  boundary  is  stripped 
away  from  the  D  effective  dimensions.  Note  that  this  set  is  compact  and 
non-empty  whenever  e  <  psym  (where  pSym  is  defined  in  Section  5.1.2). 
The  near-optimal  operating  point  ^Ty^(e)^  is  defined  as  a  solution  to 
the  following  optimization  problem:3 

Maximize  :  (5.31) 

n,c 

Subject  to:  €  Ae 

0  <  rffl  <  \nc  for  all  (n,  c) 

This  optimization  differs  from  the  optimization  in  (5.4)-(5.6)  in  that 
the  set  A  is  replaced  by  the  set  Ae. 


Lemma  5.5.  (Continuity  of  Near-Optimal  Solutions)  If  utility  func¬ 
tions  gn\r)  are  non-negative  and  concave,  and  if  there  is  a  positive 
scalar  pSym  such  that  (nsyml £  A,  then: 

E».(.C>  (4W(<0)  -  EftP  “s  e  -  0.  (5.32) 

n,c  n,c 

where  is  the  optimal  solution  of  (5.4)-(5.6). 

Proof.  The  proof  uses  convexity  of  the  capacity  region  A,  and  is  given 
in  Chapter  5.5.2  of  [108].  □ 

5.3.4  Derivation  of  Theorem  5.1 

The  proof  of  Theorem  5.1  relies  on  the  following  two  lemmas. 


Lemma  5.6.  If  the  storage  reservoirs  for  all  active  input  streams  are 
infinitely  backlogged,  then  for  any  e  such  that  0  <  e  <  nsym,  the  flow 


3  Note  that  the  final  constraint  <  (A)!3  j  is  satisfied  automatically  in  the  case  of  infi- 

nite  traffic  demand.  We  include  the  constraint  here  as  this  optimization  is  also  important 
in  the  treatment  of  general  traffic  matrices  in  Section  5.4.2. 
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control  algorithm  of  CLC1  yields: 

*CLC\U(t))  >  (<(C)W)  -  E  Uic\t)r<c\e). 

n,c  n,c 

where  ^riE(e)^  is  the  optimal  solution  of  problem  (5.31). 


Lemma  5.7.  If  the  topology  state  S(t)  is  i.i.d.  over  timeslots,  then 
for  any  e  such  that  0  <  e  <  Usym >  allocating  resources  and  routing 
according  to  CLC1  yields: 

$CLC\U(t))  >  E  Uic\t)  (r<c\e)  +  e)  ,  (5.33) 

n,c 

where  (r)E(e)^  is  the  optimal  solution  of  problem  (5.31). 


Lemma  5.6  follows  because  the  flow  control  algorithm  of  CLC1 
maximizes  ' 3/(U(t )),  defined  in  (5.27),  over  all  valid  resource  alloca¬ 
tion  options,  including  the  particular  choice  Rn'1  (t)  =  r*jc> (e)  for  all 
(n,  c).  This  is  a  valid  choice  because:  1)  all  reservoirs  are  assumed  to 
be  infinitely  backlogged,  so  there  are  always  r*nC\e)  units  of  data  avail¬ 
able,  and  2)  ^cr,E(e)  <  R™ax  (because  (r^(e))  G  A  and  hence 
Ylcr*^C'> (e)  —  Rmaxn  is  required).  Lemma  5.7  follows  because  the  re¬ 
source  allocation  and  routing  algorithm  of  CLC1  maximizes  <3?(l7(f)), 
defined  in  (5.28),  over  all  other  options,  including  the  stationary  ran¬ 
domized  strategy  of  Corollary  3.9  that  would  yield  for  all  (n,  c)  G  V: 


IE 


i  m 


=  r*(c)(e)  +  el(c) 

'  n  Vc/  ’  CJ-n  • 


(5.34) 
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Plugging  the  bounds  of  Lemmas  5.6  and  5.7  directly  into  the  drift 
expression  (5.29)  yields  the  following  for  algorithm  CLC1: 

a  m))  -  ^EE{sic)  (4c)w)  i  m}<  nb 

n,c 

-£  U<f\t)(r'n^(e)  +  / 

n,c 

-v  £><,'>  (<(c)(£))  +  ’£u!‘HtyA\c). 

n,c  n,c 

Canceling  common  terms  yields: 

A  (U(t)) (4c)w)  \U(t)  }  <  NB 

n,c 

n,c  n,c 

The  above  drift  expression  is  in  the  exact  form  specified  by  Theorem 
5.4.  Thus,  network  congestion  satisfies: 

E  Clc)  <  (NB  +  VGmax)/e,  (5.35) 

n,c 

and  time  average  performance  satisfies: 

E 9n] (r™)  >  E 9{nC)  (<(c) to)  -  NB/V.  (5.36) 

n,c  n,c 

The  performance  bounds  in  (5.35)  and  (5.36)  hold  for  any  value 
e  such  that  0  <  e  <  psym-  However,  the  particular  choice  of  e  only 
affects  the  bound  calculation  and  does  not  affect  the  CLC1  control 
policy  or  change  any  sample  path  of  system  dynamics.  We  can  thus 
optimize  the  bounds  separately  over  all  possible  e  values.  The  bound  in 
(5.35)  is  clearly  minimized  as  e  — >•  psyrn ,  yielding:  )T)n  c  <  (NB  + 
V Gmax ) / Psym .  Conversely,  the  bound  in  (5.36)  is  maximized  by  taking 
a  limit  as  e  ->  0,  yielding  by  (5.32):  Y,n,c9n\rnc)  >  Y.n,c9n)  (r,*(c))  - 
NB/V.  This  proves  Theorem  5.1.  □ 
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5.3.5  Notes 

•  The  utility  optimization  problem  for  network  flow  control 
was  first  formalized  by  Kelly  et.  al.  in  [70]  [71]  for  wireline 
networks  with  fluid-like  dynamics.  There,  network  flows  are 
described  according  to  primal  and  dual  convex  programs, 
and  Lagrange  multipliers  are  interpreted  as  shadow  prices 
that  facilitate  distributed  control  mechanisms.  The  propor¬ 
tionally  fair  objective  is  considered  in  [71]  [70],  and  related 
work  in  [100]  [107]  treats  different  systems  and  objectives. 
Game  theory  aspects  of  network  fairness  are  considered  in 
[100]  [65],  auction  algorithms  are  considered  in  [138],  and 
adversarial  queueing  theory  approaches  are  considered  in  [4] . 
The  relationship  between  utility  optimization,  convex  dual¬ 
ity  theory,  and  classical  TCP  protocols  for  wireline  networks 
is  explored  in  [97]. 

•  Convex  programming  approaches  to  static  wireless  networks 
are  considered  in  [101]  [165]  [101]  [16]  [66]  [84]  [92]  [79]  [31] 
[35].  The  work  in  [31]  investigates  systems  with  transmission 
rates  that  depend  logarithmically  on  SI  NR.  Under  the  as¬ 
sumption  that  log(l  +  SINR )  ss  log (SINR),  a  set  of  decou¬ 
pled  algorithms  for  flow  control,  routing,  and  resource  alloca¬ 
tion  are  constructed  and  shown  to  drive  resources  to  a  fixed 
optimal  operating  point.  The  work  in  [35]  considers  the  case 
log(l  +  SINR)  ~  SINR.  In  this  case,  it  is  shown  that  fixed 
operating  points  are  sub-optimal,  and  that  optimal  strategies 
involve  a  time- varying  link  transmission  schedule.  A  similar 
problem  is  investigated  in  [77],  where  NP-Completeness  re¬ 
sults  are  developed  for  transmission  scheduling. 

•  Utility  optimization  and  proportional  fairness  are  also 
treated  in  [156]  [81]  [158]  [45]  for  stochastic  wireless  down¬ 
links  with  infinite  backlog.  Fairness  for  systems  with  different 
downlink  channels  for  each  arriving  packet  is  treated  in  [24] . 

•  Fair  allocation  according  to  the  maxmin  rule  has  been  con¬ 
sidered  in  [150]  [133]  where  optimal  scheduling  policies  were 
proposed  for  single  hop  and  multihop  traffic  respectively. 
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•  The  CLC1  policy  was  developed  for  general  stochastic  net¬ 
works  in  [108]  [119].  We  note  that  alternative  approaches  to 
stochastic  network  optimization  have  recently  been  consid¬ 
ered  in  [45]  [136]  using  fluid  limit  models,  and  in  [83]  using 
stochastic  gradient  theory  (see,  for  example,  [68]).  The  Lya¬ 
punov  optimization  technique  of  this  chapter  is  related  to  the 
theory  of  static  and  stochastic  gradients,  as  the  drift  metric 
(5.25)  is  analogous  to  an  iterative  gradient  projection  for  a 
static  convex  program  (see  Chapters  4. 7-5. 7  of  [108]). 

5.4  Flow  Control  for  Arbitrary  Input  Rates 

Here  we  consider  the  general  flow  control  problem  (5.4)-(5.6),  without 
the  infinite  backlog  assumption.  The  transport  layer  storage  buffers  are 
assumed  to  have  either  infinite  or  finite  capacity  (possibly  zero).  In  the 
special  case  of  a  size-zero  storage  reservoir,  all  data  that  is  not  immedi¬ 
ately  admitted  to  the  network  layer  is  necessarily  dropped.  Let  Ln\t ) 
represent  the  current  backlog  of  commodity  c  data  in  the  transport 
layer  storage  reservoir  at  node  n  (where  L$  (t)  =  0  for  all  t  in  the  case 
of  a  size-zero  storage  reservoir).  Flow  control  decisions  are  now  sub¬ 
ject  to  the  additional  scheduling  constraint  Rn\t)  <  Ln\t)  +  An\t). 
This  constraint  is  particularly  challenging,  as  it  varies  with  time  and 
so  the  stationary  algorithm  (t)  =  r*}0'1  (e)  cannot  be  used  as  a  valid 
comparison  every  slot. 

Assume  that  the  An\t)  arrivals  are  i.i.d.  over  timeslots  with  arrival 
rates  A^  =  E|A^(t)|.  It  can  be  shown  that  for  any  matrix  (A^) 
(either  inside  or  outside  the  capacity  region),  modifying  the  CLC1  al¬ 
gorithm  to  maximize  (5.7)  subject  to  the  additional  reservoir  backlog 
constraint  yields  the  same  performance  guarantees  (5.10)  and  (5.11) 
when  utility  functions  are  linear  [108].  For  nonlinear  utilities,  such  a 
strategy  can  be  shown  to  maximize  X)ra  c  ®  ji/n'1  |  over  all 

strategies  that  make  immediate  admission/rejection  decisions  upon  ar¬ 
rival,  but  may  not  necessarily  maximize  ]> ~2ncgn  ^  |i7^(i)  ,  which 
is  the  utility  metric  of  interest. 

In  this  section,  we  solve  the  problem  by  introducing  two  new  tech- 
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niques.  The  first  is  the  use  of  auxiliary  variables  that  hold  additional 
network  flow  state  information  helpful  for  solving  nonlinear  problems 
[119].  The  second  is  the  use  of  virtual  cost  queues  that  transform  sto¬ 
chastic  constraints  involving  time  averages  into  simple  queueing  stabil¬ 
ity  problems  [119]  [136]  [115]  [111]. 

5.4.1  Problem  Transformation  via  Auxiliary  Variables 

We  begin  with  the  following  transformation  of  the  problem  (5.4)-(5.6), 
which  introduces  new  variables  7 ^  for  each  input  stream  Rn\t ): 


Maximize:  (5-37) 

n,c 

Subject  to:  rffi  >  (5.38) 

(ric))  G  A,  (5.39) 

0  <  <  a£>.  (5.40) 


It  is  not  difficult  to  show  that  the  optimal  solution  of  the  above  problem 
is  exactly  the  same  as  the  optimal  solution  of  the  original  problem  (5.4)- 
(5. 6). 4  Note  that  if  the  ry'1  variables  are  associated  with  time  averages 
of  the  flow  control  inputs  Rn\t ),  then  any  control  policy  that  stabilizes 
the  system  will  naturally  lead  to  time  averages  that  satisfy  (5.39)  and 
(5.40).  Furthermore,  the  utility  optimization  is  now  expressed  entirely 
in  terms  of  the  new  variables  7 while  the  additional  inequality  (5.38) 
expresses  a  linear  constraint  between  the  time  average  flow  control 
decisions  and  the  7 ^  variables. 

To  solve  the  above  problem,  for  each  active  input  stream  An ^  (t)  we 
define  additional  flow  state  queues  Yn\t )  that  ensure  the  additional 
constraint  (5.38)  is  satisfied.  Specifically,  we  define  1^(0)  =  0  for  all 
(n,  c),  and  update  the  flow  state  queues  every  timeslot  as  follows: 

Y^(t  +  l)  =  max[Y^(t)-R^(t),0]  +  ^(t),  (5.41) 

where  7 n\t)  represents  a  process  of  non-negative  auxiliary  variables 
that  the  flow  controller  computes  on  every  timeslot.  The  Y„c\t )  process 


4  Recall  that  the  (r)  utility  functions  are  non-decreasing. 
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Fig.  5.1  A  queueing  illustration  of  the  dynamic  equation  (5.41)  for  the  Yn°\t)  queues. 


can  also  be  viewed  as  a  virtual  queue  with  “arrivals”  7^  (t)  and  “server 
rate”  Rn\t )  (see  Fig.  5.1).  Let  Y(t )  =  (YnC\t))  represent  the  matrix 
of  these  flow  state  queues,  and  let  0(t)  =  [Y(f);  U(t)]  represent  the 
combined  matrix  of  flow  state  queues  and  actual  queues. 

To  motivate  the  control  algorithm,  suppose  that  the  Rn\t )  and 
7,^ (t)  processes  have  well  defined  time  averages  and  7 n\  respec¬ 
tively. 

Observation:  If  a  control  algorithm  stabilizes  all  actual  queues  U(t ) 
and  flow  state  queues  Y(t )  of  the  system ,  then  the  resulting  time  aver¬ 
ages  f  -c^  and  7^  must  satisfy  all  inequality  conditions  (5.38)-(5.f0). 

That  (5.39)  and  (5.40)  hold  if  the  U(t )  queues  are  stable  is  clear 
because  the  admitted  rates  must  always  be  less  than  or  equal  to  the  ac¬ 
tual  arrival  rates,  and  because  the  admitted  rate  matrix  must  be  within 
the  network  capacity  region  A  for  stability  of  the  network  queues.  The 
key  component  of  the  above  observation  is  that  the  constraints  (5.38) 
hold  if  all  virtual  queues  Y(t)  are  stable.  This  follows  from  Lemma  3.3 
and  the  basic  queueing  inequality, 

t- 1  t- 1 

+  >£#(<)• 

T— 0  T— 0 

The  above  observation  was  introduced  in  [109]  for  the  purpose  of  design¬ 
ing  optimal  scheduling  algorithms  for  wireless  networks  with  average 
power  constraints,  where  virtual  power  queues  were  used  to  transform 
stochastic  inequality  constraints  into  queueing  stability  problems.  Sim¬ 
ilar  techniques  have  recently  been  used  to  treat  other  stochastic  cost 
constraints  in  [136]  [114]  [115]. 

The  above  observation  suggests  the  approach  of  designing  a  network 
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control  algorithm  to  stabilize  all  queues  U(t)  and  Y(t)  while  maximiz¬ 
ing  the  utility  function  g(-).  To  this  end,  define  the  Lyapunov  function 
L(0)  as  follows: 

n,c  n,c 

where  t]  is  a  parameter  that  satisfies  0  <  rj  <  1  and  determines  the 
relative  weight  of  virtual  queues  in  the  control  problem.  It  turns  out 
that  choosing  r/  small  decreases  the  time  average  backlog  in  the  actual 
queues  while  increasing  the  time  average  backlog  in  the  virtual  flow 
state  queues.  The  actual  queues  determine  the  actual  congestion  and 
delay  in  the  network,  while  the  virtual  queues  play  a  role  in  determin¬ 
ing  the  transient  time  or  “learning  time”  required  for  the  system  to 
approach  optimal  performance. 

The  conditional  Lyapunov  drift  is  given  by: 

A (©(*))4e  {L{@(t  +  1))  -  L(G{t))  1 0(t) }  . 

Motivated  by  the  drift  condition  of  the  Lyapunov  Optimization  Theo¬ 
rem  (Theorem  5.4),  we  design  a  control  policy  to  minimize  the  following 
metric: 

Minimize:  A (0(f))  -  FIE  (7 £%))  l0W 

l  n,c 

5.4.2  The  Cross  Layer  Control  Algorithm 

To  simplify  exposition,  suppose  that  the  exogenous  arrivals  to  a  given 
node  n  are  deterministically  bounded  by  a  constant  Rl7llLax  every  times¬ 
lot,  so  that: 

J2A{n\t)  <KaX  for  all  t. 

C 

Further,  we  shall  use  the  “Type  2”  flow  control  constraints,  so  that: 
Rn\t)  <  min  Ln\t )  +  > 

"(c). 

where  R\ ;  are  suitably  large  constants  that  satisfy: 

A$(t)<R$  for  all  t 
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(note  that  choosing  =  R™ax  for  all  n  ensures  the  above  inequality  is 
satisfied).  This  “Type  2”  flow  control  constraint  simplifies  the  algorithm 
at  the  expense  of  increasing  the  average  congestion  bound. 

Using  the  queueing  equations  (5.41)  and  (5.3),  the  following  drift 
bound  can  be  computed: 


A(0(f))  -  UE  |  X]  g{nc)  (t | ©(*)  1  <  c*2 


n,c 


n,c 

+ 


|©(t) 


Ewd-E^w 

L  b  a 

V  c/(c)  ( t)E  i  R^c)  (t)  1 0  (t)  \ 

n,c 

V  E  yic)(i)E  {  [R{n\t)  -  1 0(t)  } 


-uE|E<?ic) 

\ 


(5.42) 


where  C2  is  a  constant  that  depends  on  ^^aa.  n,  R°nax  n-  an<i  Rn  ^  Choos¬ 
ing  the  control  decision  variables  to  minimize  the  right  hand  side  of  the 
above  inequality  leads  to  the  following  Cross  Layer  Control  algorithm:5 

Cross  Layer  Control  Algorithm  (CLC2b):  Every  timeslot,  the 
topology  state  S(t)  and  the  queue  values  U(t),  Y(t)  are  observed,  and 
controllers  perform  the  following  actions: 


(1)  Flow  Control:  Every  timeslot  and  for  each  active  input 
stream  (n, c),  observe  Un\t)  and  Yn,c\t)  and  choose: 


R[n\t)  = 


J  min 

to 

if  r]Y^c)  (t) 
otherwise 


r(c)/ 


Furthermore,  for  each  (n,  c),  choose  7 n\t)  =  7 n  \  where  7^ 


5  This  control  algorithm  is  a  modified  version  of  the  original  CLC2  algorithm  from  [119], 
and  hence  we  label  it  “CLC2b.” 
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solves: 

Maximize:  V  'g$  (7)  -  'rtYn)  (t)l, 

Subject  to:  0  <  7  <  R$. 

The  flow  state  queues  Yn°\t)  are  then  updated  according  to 
(5.41). 

(2)  Routing  and  Resource  Allocation:  Same  as  the  dynamic 
backpressure  algorithm  of  CLC1. 

The  following  theorem  assumes  arrival  matrices  A(f)  and  topology 
states  S(t)  are  i.i.d.  over  timeslots,  and  assumes  that  Gmax  is  a  para¬ 
meter  such  that  J2n.c !Jn}  (Rn\t))  <  Gmax  for  all  t. 


Theorem  5.8.  For  arbitrary  rate  matrices  ^An^  (possibly  outside  of 
the  capacity  region),  for  any  V  >  0,  and  for  any  reservoir  buffer  size 
(possibly  zero),  the  CLC2b  algorithm  stabilizes  the  network  and  yields 
the  following  congestion  and  utility  bounds: 


lim  sup  - 


C2  +  V  Gn 


t— >00 


t= 0  n,c 


f^sym 


q(c)  (r*(c)\  _ 
Vn  \  1  n 


c2 

V  ' 


(5.43) 

(5.44) 


Proof.  The  proof  follows  because,  given  a  particular  queue  state  0(f)  = 
[17(f),  Y(t)]  at  time  f,  the  CLC2b  algorithm  maximizes  the  right 
hand  side  of  (5.42)  over  all  alternate  choices  of  the  decision  variables 
Rn\t) , (f) ,  It  is  not  difficult  to  construct  alternate  polices 

that  choose  R^c\t),  R*a^\t),  7n^(f)  to  yield  for  all  (n,  c)  €  V  (see 
[119]): 


E  (*)  -  E  t^\t)  m)  |  =  r<c\e)  +  eltf 

7;(c)(f)  =  r<c\  E  {R<c\t)  [0  (f)  }  =  r*n^ 
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Plugging  these  particular  decision  variables  into  the  right  hand  side  of 
(5.42)  thus  preserves  the  inequality  and  creates  many  terms  that  can 
be  cancelled,  yielding: 

A (©(i))  -  VT&  jx>ic)  |0(t)  \<C2 

-e^£/(c)(t)-R^(c)  (<(C)(£))- 

n,c  n,  c 

The  above  inequality  is  in  the  exact  for  for  application  of  the  Lya¬ 
punov  Optimization  Theorem  (Theorem  5.4),  and  holds  for  any  value 
e  such  that  0  <  e  <  /is ym.  Applying  the  theorem  and  optimizing  over 
all  e  shows  that  the  queues  U(t )  are  strongly  stable  with  the  conges¬ 
tion  bound  (5.43),  and  that  a  utility  bound  similar  to  (5.44)  holds. 
A  similar  argument  can  be  used  to  prove  strong  stability  of  the  Y(t) 
queues,  which  relates  this  utility  bound  directly  to  (5.44)  and  proves 
the  theorem  (see  [119]  for  details).  □ 

We  note  that  if  the  constants  R™ax  and  wn  are  chosen  to  be  equal 
to  some  fixed  constant  R,  then  the  C2  constant  in  Theorem  5.8  is 
O(MR),  where  M  is  the  number  of  active  sessions  (n,  c)  throughout 
the  network.  The  constant  can  be  reduced  by  replacing  the  Type  2 
flow  control  constraints  with  Type  1  constraints,  so  that  every  times¬ 
lot  and  for  each  node  n  the  Rn\t )  variables  are  chosen  to  maxi¬ 
mize  Y^c=l  Rn\t)[vYnC\t)  -  Un\t)\  Subject  to  J2cRn\t)  <  R,  0  < 
Rn\t)  <  Ln\t)+An\t).  The  solution  of  this  optimization  is  still  quite 
simple,  as  it  amounts  to  admitting  as  much  data  as  possible  from  the 
commodities  c  with  the  largest  (positive)  values  of  rjYnC\t )  —  Un\t). 
This  modification  would  yield  a  constant  C2  that  is  +  N)R), 

which  can  be  pushed  to  0(NR)  by  choosing  ij  appropriately  small. 

The  average  backlog  in  the  virtual  queues  can  also  be  decreased  if, 
for  each  source  node  n,  the  7 n\t)  optimization  were  replaced  by  the 
more  complex  optimization  of  maximizing  ^2cgn\'7n^)  subject  to  the 
simplex  constraint  7 ^  >  0,  )T)C  <  R.  In  this  case,  we  could  choose 
77  =  1  and  still  have  C2  =  O(NR).  Reducing  the  average  virtual  queue 
backlog  is  advantageous  as  it  reduces  the  time  required  for  the  network 
to  adapt  to  possible  changes  in  traffic  rates  or  channel  statistics. 
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5.4.3  An  Alternative  Construction  on  Flow  State  Queues 

Our  presentation  of  the  CLC2b  algorithm  is  somewhat  different  than 
the  original  CLC2  algorithm  developed  in  [119].  The  original  CLC2 
algorithm  used  Type  1  flow  control  constraints,  and  used  a  different 
transformation.  Specifically,  for  each  utility  function  gif'*  a  new 

function  hif  was  formed: 

(a^A -a^(Rin  1  -  o(c)  (  Rin  -  a(c)N] 

,bn  \^n  J  — 2m  x^^maxJ  tfn  y^^max  n  j  ? 

where  af\t)  represents  a  process  of  auxiliary  variables.  Letting  rff* 
represent  the  time  average  of  the  flow  control  decisions  Rn\t)  and 
letting  af'*  represent  the  time  average  of  the  auxiliary  processes  an*  ( t ) , 
we  observe  the  following: 

Minimizing  J2nchlf  subject  to  network  stability  and  to  the 

additional  constraint  off*  >  Rlfax  ~  ^if*  for  all  (n,  c)  is  equivalent  to 
maximizing  Yfnc9n'>  su^jec^'  t°  network  stability. 

To  ensure  that  af*  >  Rffax  —  rff ,  the  algorithm  of  [119]  uses  a  set 
of  flow  state  queues  Yn\t )  with  update  equations  as  follows: 

Y^\t  +  1)  =  max  [yic)(t)  -  «ic)(t), 0]  +  Rfax  ~  (5.45) 

The  CLC2  algorithm  performs  resource  allocation  and  routing  in 
the  same  manner  as  in  CLC2b,  but  chooses  flow  control  values  Rff  {t ) 
for  each  source  node  n  to  maximize  rl^n  '>  (i)  —  uf*  ( t )  Rif*  ( t ) 

subject  to  the  Type  1  flow  control  constraints.  The  auxiliary  vari¬ 
ables  af*  (t)  are  then  computed  by  maximizing  V gif'*  (^R'fax  —  af*^j  + 

r/Yfc\t)aif  subject  to  0  <  af*  <  R™ax- 

5.4.4  Notes 

•  Simulation  results  of  the  CLC2  algorithm  for  downlinks, 

N  x  N  packet  switches,  and  multi-hop  networks  are  found  in 

[119]- 

•  The  method  of  introducing  auxiliary  variables  and  flow  state 
queues  to  solve  the  flow  control  problem  was  developed  in 
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[119].  The  transformation  (5.37)-(5.40)  was  later  considered 
in  [111]  [115],  and  a  related  use  of  auxiliary  variables  is  pre¬ 
sented  in  [94],  An  alternative  approach  to  utility  optimiza¬ 
tion  is  developed  in  [136]  using  fluid  model  transformations. 
It  is  interesting  to  note  that  the  algorithm  in  [136]  also  keeps 
additional  variables  to  solve  the  nonlinear  optimization  prob¬ 
lem,  similar  to  the  auxiliary  variables  and  flow  state  queues 
developed  in  this  chapter.  A  more  direct  comparison  of  the 
two  methods  is  presented  in  the  next  chapter. 

The  [0(1/V),0(V)\  utility-delay  tradeoff  achieved  by 
CLC2b  is  not  the  optimal  tradeoff.  A  recent  result  in  [115] 
demonstrates  that  an  improved  tradeoff  [0(1 /V),  0(log(V))] 
is  achievable  using  a  more  sophisticated  flow  control  algo¬ 
rithm.  Further,  [115]  shows  that,  for  the  special  case  of  one- 
hop  networks,  this  logarithmic  structure  of  the  utility-delay 
tradeoff  cannot  be  improved  by  any  alternative  control  strat¬ 
egy.  Related  work  in  [15]  [114]  considers  the  fundamental 
energy-delay  tradeoff  for  single-user  and  multi-user  wireless 
systems,  where  a  square-root  tradeoff  law  is  established.  En¬ 
ergy  optimal  networking  is  considered  in  more  detail  in  the 
next  chapter. 

When  the  exogenous  traffic  arrival  rates  are  outside  the  ca¬ 
pacity  region,  it  may  be  of  interest  to  control  the  rate  of 
increase  of  node  queues  in  order  to  delay  buffer  fill-up  as 
long  as  possible.  It  turns  out  that  the  dynamic  backpressure 
policy  has  desirable  properties  in  this  respect  as  well.  The 
study  of  this  problem  for  networks  with  fixed  link  capacities 
is  presented  in  [52],  For  networks  with  changing  link  capac¬ 
ities  the  problem  can  be  dealt  with  using  the  methodologies 
developed  in  this  paper.  In  this  case,  instead  of  flow  control  of 
exogenous  traffic  each  of  the  nodes  of  the  network  must  exer¬ 
cise  control  of  its  queues  by  adding  extra  “overflow”  buffers. 
A  step  towards  this  direction  has  been  taken  in  [53]. 
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Here  we  use  Lyapunov  optimization  theory  to  develop  a  framework  for 
optimizing  stochastic  networks  with  general  cost  and  reward  metrics. 
The  results  in  this  chapter  are  well  suited  to  solve  problems  of  energy 
optimal  networking ,  including  problems  of  minimizing  average  power 
expenditure  in  mobile  ad-hoc  networks,  and  problems  of  maximizing 
network  throughput  utility  subject  to  average  power  constraints.  The 
general  solution  to  these  problems  integrates  the  basic  Lyapunov  stabil¬ 
ity  and  optimization  concepts  developed  in  previous  chapters,  including 
the  use  of  auxiliary  variables  and  virtual  cost  queues. 

6.1  The  Network  Model  Assumptions 

We  consider  the  same  network  as  in  the  previous  chapter,  with  N  nodes, 
K  commodities  (denoted  by  a  node  set  M  and  a  commodity  set  1C), 
a  topology  state  process  S(t),  and  a  link  transmission  rate  function 
C(I(t),  S(t)).  Exogenous  arrivals  are  given  by  A(t)  =  ( An\t )).  For 
simplicity,  we  continue  to  assume  that  the  pair  [S'(t);  A(f)]  is  i.i.d.  over 
timeslots,  with  the  understanding  that  similar  results  can  be  extended 
to  non-i.i.d.  systems  using  T-slot  analysis. 
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Control  decision  variables  I (t) ,  ( t ) ,  and  ( t )  for  resource  al¬ 

location,  routing,  and  flow  control  are  also  the  same  as  in  previous 
chapters,  with  the  exception  that  we  restrict  attention  to  the  “Type  2” 
flow  control  constraints  (see  Section  5.1)  for  simplicity,  so  that 

"(c) 

for  suitably  large  constants  Rn-  Specifically,  for  simplicity  we  continue 

~  (c) 

to  assume  that  arrivals  are  deterministically  bounded,  and  choose  Rn 
so  that  An\t )  <  Rn ^  for  all  t.  The  I(t)  decisions  satisfy  I(t)  G  %s(t) 
for  all  t.  and  the  (t)  variables  satisfy  (5.1)  and  (5.2).  Transport 
layer  storage  reservoirs  have  arbitrary  storage  space  (infinite,  finite,  or 
zero).  Network  layer  queueing  dynamics  are  given  in  (5.3)  of  Chapter  5. 
This  general  framework  can  also  be  used  to  treat  networks  without  flow 
control  by  simply  adding  the  additional  constraint  An  '1  (t)  =  Rn  ^  (t)  for 
all  t.  Recall  that  the  sets  Cc  restrict  routing  decisions,  and  the  set  V 
consists  of  all  (n,  c)  pairs  for  which  there  is  a  valid  queue  Un\t). 

6.1.1  Network  Penalties  and  Rewards 

Let  x(t)  =  . . . ,  xmx  (t))  represent  a  vector  of  Mx  penalties  in¬ 
curred  by  the  network  control  decisions  I(t),  (t).  Rn\t)  at  timeslot 

t,  and  let  y(t)  =  (yi(t), . . .  ,VMy{t ))  represent  a  vector  of  My  rewards 
earned  at  timeslot  t  (where  Mx  and  My  are  arbitrary  integers).  For 
example,  in  a  network  with  power  allocation  decisions,  a  penalty  xm(t) 
might  represent  an  arbitrary  function  of  the  power  expended  at  one  or 
more  nodes  during  slot  t.  such  as: 

•  xm{t )  =  P-nbit)  for  a  given  node  n  G  M  associated  with 
penalty  m. 

•  Xm(t)  =  (Pnb(t))2  +  Pab(t)Pc,d(t)  for  a  given  node  n  G  M 

and  some  given  links  (a,  6),  (c,  d)  associated  with  penalty  m. 

•  xm(t)  =  ep°6^  +  /x^(t)  for  some  link  (a,  h)  and  some  com¬ 
modity  c  associated  with  penalty  m. 

Likewise,  a  reward  ym(t)  can  be  defined  arbitrarily,  and  is  usu¬ 
ally  associated  with  the  flow  control  variables  Rn\t).  This  abstract 
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use  of  network  penalties  was  introduced  in  [136],  where  a  fluid  model 
transformation  was  used  to  analyze  performance  (the  result  of  [136]  is 
considered  in  more  detail  in  Section  6.4).  Here,  we  demonstrate  how 
these  general  penalty  and  reward  functions  can  be  treated  using  the 
Lyapunov  optimization  techniques  developed  in  the  previous  chapters. 

We  assume  that  penalties  and  rewards  are  non-negative  and  upper 
bounded  by  positive  vectors  Xmax  and  Y™ax,  so  that  0  <  x(t)  <  Xmax 
and  0  <  y(t)  <  Yinax  for  all  time  t  (inequalities  taken  entrywise). 
Let  x  =  (xi , ,  xmx  )  and  y  =  (y1, ... ,  yMy )  represent  the  vector  of 
time  average  penalties  and  rewards  (assuming  for  now  that  such  time 
averages  exist).  We  consider  the  problem  of  minimizing  the  sum  of  a 
convex  increasing  function  of  the  long  term  average  penalty  vector  and 
a  convex  decreasing  function  of  the  long  term  average  reward  vector, 
subject  to  network  stability  and  also  subject  to  an  additional  set  of 
convex  constraints  on  the  long  term  average  penalties  and  rewards. 

Specifically,  let  f(x )  represent  a  scalar  valued  cost  function  as¬ 
sociated  with  the  penalty  vector  x.  We  assume  that  f(x)  is  non¬ 
negative,  continuous,  convex  in  the  multi-dimensional  vector  x ,  and 
entrywise  non-decreasing  (so  that  f(x)  <  f(y)  whenever  x  <  y, 
where  inequality  is  taken  entrywise).  We  assume  there  is  a  value  fmax 
such  that  f(x(t))  <  fmax  for  all  t.  Let  the  vector  valued  function 
q(x)  =  (qi(x), . . . ,  qjx(x))  represent  an  additional  set  of  cost  func¬ 
tions,  where  each  component  function  q3  ( x )  is  similarly  non-negative, 
continuous,  bounded,  convex,  and  entrywise  non-decreasing. 

Similarly,  we  let  g(y),  h(y)  =  (hi(y), . . .  ,hjy(y)),  represent  utility 
functions  associated  with  the  rewards  y.  These  functions  are  assumed 
to  be  non-negative,  continuous,  bounded,  concave,  and  entrywise  non¬ 
decreasing.  The  generalized  stochastic  optimal  networking  problem  is: 

Minimize:  f(x)  —  g(y )  (6-1) 

Subject  to:  1)  q(x)  <  Q, 

2)  h(y)  >  H, 

3)  Network  Stability, 

where  Q  =  (Q i, . . . ,  Qjx)  is  a  vector  of  required  upper  bounds  on  the 
cost  functions  q(x),  and  likewise  H  is  a  vector  of  lower  bounds  on  the 
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utility  functions  h[y).  We  define  A q  as  the  modified  capacity  region  for 
this  network,  consisting  of  all  arrival  rate  matrices  that  can  be  stably 
supported  by  the  network  layer  under  the  additional  cost  constraints 
q(x)  <  Q,  h(y)  >  H  imposed  on  the  network  penalties  and  rewards. 

Note  that  the  general  problem  can  be  stated  purely  in  terms  of 
penalties.  Indeed,  the  reward  objective  is  equivalent  to  minimizing  the 
function  f(x(t)),  where  —  y(t),  and  where: 

f(x(t))=9(  Y™* )  -  g(  Y™a*  -  x(t)).  (6.2) 

Because  g(y)  is  concave  and  entry  wise  non-decreasing,  the  function 
f(x)  is  convex  and  entrywise  non-decreasing,  and  therefore  fits  into 
our  general  framework.  This  transformation  is  used  to  treat  the  prob¬ 
lem  of  fair  flow  control  in  [119]  (as  discussed  in  Chapter  5,  Section 
5.4.3).  Likewise,  penalties  can  be  changed  into  rewards  through  a  simi¬ 
lar  transformation.  However,  it  is  often  useful  to  maintain  a  distinction 
between  penalties  and  rewards,  especially  for  implementation  purposes. 


6.1.2  Assumptions  on  Optimal  Stationary  Control 

Let  c*  =  f*  —  g*  represent  the  optimal  cost  associated  with  the  problem 
(6.1),  and  let  x* ,  y*  represent  the  optimal  time  average  penalty  and 
reward  vectors,  so  that  f(x*)  =  /*,  g(y*)  =  g* ■  We  consider  the  class 
of  systems  for  which  optimality  can  be  achieved  within  the  class  of 
S-only  algorithms,  defined  as  follows: 


Definition  6.1.  An  S-only  algorithm  is  a  stationary  randomized  al¬ 
gorithm  that  chooses  control  variables  I(t),  {Rn\t)},  {/z^ (t)}  based 
only  on  the  current  topology  state  S(t). 


While  our  dynamic  control  algorithm  will  base  decisions  on  current 
queue  backlog  and  hence  is  not  S-only,  its  performance  will  be  evaluated 
by  comparison  to  S-only  algorithms.  Specifically,  we  assume: 

Assumption  1:  (Optimality  of  an  S-only  Policy)  There  exists  an  S'- 
only  algorithm  such  that  for  all  t  and  all  node-commodity  pairs  (n,  c)  G 
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T>  we  have: 

£  E  {til  it) }  +  E  {/?4C)  (0  }  =  £  E  {mS  (*) }  ,  (6.3) 

a  b 

q(®{x(t)})<Q , 

/  (E{®(*)})  -  g(E  {y(t)})  =  f*-  g*,  (6.4) 

where  the  expectations  above  are  taken  over  the  random  topology  state 
S(t)  and  the  potentially  randomized  control  decisions. 

Further,  we  make  the  following  interior  point  and  slackness  assump¬ 
tions  (which  hold  for  all  of  the  networks  considered  in  this  paper): 

Assumption  2:  (Interior  Point)  There  exists  a  value  emax  >  0  to¬ 
gether  with  an  S'-only  algorithm  such  that  for  all  t  and  all  node¬ 
commodity  pairs  (n,  c)  £  V  we  have: 

£  E  [til (; t ) }  +  emax  +  E  [r^ (t) }  =  £  E  (t) }  , 

a  b 

q(E{x(t)})<Q>  h(M{y(t)})>H. 

Assumption  3:  (Slackness  1)  There  exist  positive  values  61,62  to¬ 
gether  with  an  5-only  algorithm  that  yields  (6.3)  and  that  simultane¬ 
ously  yields: 

q(E  (®(i)})  <  Q  -  6-ilq,  /i(E  {y(t)})  >  H  +  621h- 

where  lq  is  a  vector  with  all  entries  equal  to  0  or  1,  with  entry  i  equal 
to  1  if  and  only  if  function  qi(-)  is  not  identically  zero,  and  1/,  is  a  0/1 
vector  with  entry  j  equal  to  1  if  and  only  of  h3  ( • )  is  not  identically  zero. 

Assumption  (Slackness  2)  There  exist  positive  values  <53,^4  and 
vectors  a,  7  such  that  q(a)  <  Q,  h( 7)  >  H  and  0  <  a  <  Xmax , 
0  <  7  <  Ymax,  such  that  there  exists  an  S’-only  algorithm  yielding 
(6.3)  while  simultaneously  yielding: 

E{*(t)}  <  a  —  531x,  E{y(i)}  >  7  +  64ly. 

where  lx  is  a  0/1  vector  with  entry  i  equal  to  1  if  and  only  if  penalty 
Xi(t )  is  not  identically  zero  for  all  t,  and  ly  is  a  0/1  vector  with  entry 
j  equal  to  1  if  and  only  if  reward  yj(t)  is  not  identically  zero  for  all  t. 

Note  that  the  quantities  (t)}  can  be  viewed  as  flows  , 

and  hence  the  control  decisions  of  the  S'-only  policy  of  Assumption 
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1  yield  flows  that  satisfy  the  flow  conservation  constraints,  while  also 
satisfying  the  cost  and  utility  inequality  constraints  and  yielding  time 
average  penalties  and  rewards  that  achieve  the  minimum  network  cost 
f(x)  —  g(y)  =  f*  —  g*  ■  The  value  c*  =  f*  —  g*  can  thus  be  interpreted 
as  the  optimal  cost  over  all  S'-only  policies.  For  a  large  class  of  network 
flow  problems,  including  the  fairness  problems  of  the  previous  chapter 
and  the  energy  problems  to  be  presented  in  this  chapter,  c*  is  also  the 
optimal  cost  over  all  possible  (perhaps  not  S'-only)  policies  [109] . 

Assumption  2  effectively  states  that  there  exists  an  S-only  policy 
that  meets  all  inequality  constraints  while  supporting  a  traffic  matrix 
(E{i?nC)(£)}  +  elnC))  (where  is  equal  to  1  if  (n,  c)  €  T>,  and  zero  else). 
Suppose  S-only  policies  can  be  used  to  support  any  traffic  matrix  within 
the  relative  interior  of  the  network  capacity  region  A q,  and  define  nSym 
as  the  edge  size  of  the  largest  U-dimensional  hypercube  that  can  fit 
inside  the  effective  dimensions  of  A q  (with  one  vertex  at  the  origin). 
If  ptsym  >  0  and  if  the  functions  q(x)  and  h(y)  do  not  depend  on 
penalties  or  rewards  associated  with  the  flow  control  decision  variables 
Rn\t ),  then  Assumption  2  is  satisfied.  This  can  be  seen  by  setting 
Rn\t )  =  0  for  all  (n, c)  and  all  t,  so  that  emax  is  equal  to  gSym-  In  the 
case  when  there  are  no  flow  controllers  (so  that  An\t)  =  Rn\t )  for 
all  (n,  c)  and  all  t),  the  value  of  emax  is  the  largest  value  of  e  such  that 
(A,ic)  +  elic))  €  Aq. 

Assumption  3  states  that  there  exists  an  S'-only  algorithm  that 
satisfies  the  flow  constraints  while  also  yielding  the  strict  inequalities 
qi  (IE  {as(t)})  <  Qi  and  /i*(1E  {y(t)})  >  Hi  whenever  the  corresponding 
qi(-)  and  hi(-)  functions  are  not  identically  zero  and  hence  define  legiti¬ 
mate  constraints.  Assumption  4  is  similar.  In  many  networks  with  flow 
control  and/or  without  cost  or  utility  constraint  functions  </(•),  h(-),  as¬ 
sumptions  3  and  4  can  be  shown  to  trivially  hold  simply  by  considering 
the  policy  that  sets  all  control  variables  to  zero  or  near-zero. 

6.1.3  Example:  A  Wireless  Uplink  with  Power  Constraints 

Consider  a  wireless  uplink  where  L  users  transmit  to  a  bases- 
tation  over  L  time  varying  channels.  Let  C(P(t),  S(t))  = 

(Ci(P(t),  S(t)), . . .  ,CL(P(t),  S(t)))  represent  the  link  transmission 
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rate  vector  as  a  function  of  the  current  topology  state  S(t)  and  the 
power  allocation  vector  P(t)  =  (P\(t), ... ,  Pfjt)).  The  particular  form 
of  the  C(P(t),  S(t ))  function  depends  on  the  physical  layer  transmis¬ 
sion  and  multi-user  detection  schemes  used  by  this  system.  Suppose 
that  power  allocation  is  restricted  so  that  0  <  P,(f)  <  Pmax  for  all 
users  i  and  for  all  t.  Separate  queues  are  maintained  at  each  user,  and 
we  let  R-i(t)  represent  the  amount  of  data  user  i  decides  to  add  to  its 
queue  on  timeslot  t.  We  assume  that  0  <  Ri(t)  <  Rmax  for  some  con¬ 
stant  Rmax ■  Suppose  that  the  transport  layer  reservoirs  at  each  user 
are  infinite  with  infinite  backlog. 

Consider  the  following  network  parameters,  with  penalties  x(t)  and 
rewards  y(t )  defined  according  to  power  allocation  and  flow  control 
decisions: 

•  x(t)  =  (Pi{t),...,PL{t)),  y(t)  =  (Ri(t), . . .  ,RL(t)) 

•  q(x)  =  x,  Q  =  (.1,  .1, . . . ,  .1)  Watts;  h(y)  =  H  =  0 

•  /(*)  -  g(v)  =  Pi  Ef=  i  X1  -  A  Ef=i  fog(!  +  yt) 

where  Pi,  @2  are  non-negative  constants  that  weight  the  relative  im¬ 
portance  of  energy  cost  and  throughput  utility.  Thus,  in  this  simple 
example,  the  penalties  x(t)  correspond  to  power  allocations  and  the 
rewards  y(t)  correspond  to  transport  layer  admission  decisions.  The 
inequality  constraint  q(x)  <  Q  ensures  that  the  average  power  ex¬ 
pended  by  each  user  is  no  more  than  0.1  Wafts.  The  global  cost  function 
f(x)  —  g(y)  contains  a  negative  throughput  utility  term  and  a  positive 
term  that  is  quadratic  in  the  power  penalties.  Such  a  quadratic  cost  on 
power  expenditure  might  be  defined  to  provide  a  measure  of  “energy 
fairness”  among  the  different  users,  so  that  the  resulting  average  power 
costs  are  balanced  more  evenly  across  users. 

It  can  be  shown  that  Assumption  1  holds  for  this  example,  and 
that  Assumption  2  holds  whenever  the  capacity  region  A q  contains  a 
positive  vector  (e, . . . ,  e)  [109].  If  C(0,S(t))  =  0  for  all  S(t )  (so  that 
zero  power  yields  zero  transmission  rate),  then  Assumption  3  trivially 
holds  by  assigning  R(t)  =  0,  P(t)  =  0,  (/x^(t))  =  (0)  for  all  t,  so  that 
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(6.3)  holds,  and 

q(E  (®(t)})  =  q{ 0)  =  0  <  Q  =  (0.1, . . . ,  0.1), 

and  hence  Assumption  3  is  satisfied  for  <5i  =  0.1.  Further,  choosing 
a  =  Q,7  =  0,  and  choosing  positive  c>3 , 84  values  such  that  there 
exists  an  S'-only  algorithm  to  support  a  throughput  vector  (84, . . .  ,84) 
subject  to  the  reduced  average  power  constraint  Q  —  8zlx  ensures  that 
Assumption  4  is  satisfied. 

6.2  Algorithm  Design 

Here  we  specify  a  dynamic  control  algorithm  that  does  not  require 
knowledge  of  traffic  statistics  or  topology  state  probabilities,  yet  meets 
all  network  constraints  while  yielding  a  total  network  cost  that  is  arbi¬ 
trarily  close  to  the  minimum  cost  c*  =  f*  —  g* ,  with  a  corresponding 
trade-off  in  average  end-to-end  network  delay.  To  motivate  the  control 
algorithm,  note  that  the  optimization  problem  (6.1)  is  equivalent  to  the 
following  modified  problem  that  introduces  new  variables  a  and  7: 

Minimize: /(a)  —  g{  7) 

Subject  to:  :  1)  q(a)  <  Q,  h{ 7)  >  H, 

2)  x  <  a,  y  >  7, 

3)  Network  Stability. 

As  in  Chapter  5,  we  have  introduced  a  new  vector  a  to  decouple 
the  penalty  variables  x  from  the  cost  functions  /(•)  and  g(-),  and  have 
similarly  introduced  a  new  vector  7  to  decouple  the  rewards  from  the 
utility  functions.  The  decoupling  vector  a  is  not  necessary  in  the  special 
case  when  cost  functions  /(•)  and  g(-)  are  both  linear.1  Likewise,  the 
vector  7  is  not  necessary  when  h(-)  and  g{-)  are  both  linear.  However, 
these  extra  vectors  are  important  for  the  treatment  of  general  (poten¬ 
tially  non-linear)  convex  cost  functions  and  concave  utility  functions. 
Let  a(t )  and  7 (t)  represent  a  process  of  these  new  vector  variables,  to 
be  chosen  as  control  parameters  on  each  timeslot.  To  ensure  that  the 

1  Using  decoupling  vectors  a  in  the  case  of  linear  costs  cannot  hurt,  but  linearity  can  be 
exploited  to  design  a  somewhat  simpler  algorithm  (see  Section  6.3). 
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first  set  of  inequality  constraints  in  the  above  problem  are  satisfied, 
we  define  virtual  cost  queues  Dq(t),  D^t),  with  update  equations  as 
follows: 


Dq(t  +  1)  =  ma x[Dq(t)  —  Q,  0]  +  q(a(t)),  (6.5) 

Dh{t  +  1)  =  max[Dh(f)  -  h(j(t)),  0]  +  H  (6.6) 

Likewise,  to  ensure  the  second  set  of  inequality  constraints  are  satis¬ 
fied,  we  define  virtual  queues  Zx(t )  and  Zy(t),  which  are  generalized 
versions  of  the  flow  state  queues  developed  in  the  previous  chapter,  and 
have  update  equations: 


Zx(t  +  1)  =  max[Zx(t)  —  a(t ),  0]  +  x(t),  (6-7) 

Zy(t  +  1)  =  max[ZJ/(t)  -  y(t),  0]  +  7 (t).  (6.8) 


Note  that  the  Dq(t)  variables  can  be  viewed  as  backlogs  in  a  virtual 
queue  with  input  process  q(a(t))  and  constant  service  rate  Q.  Thus,  if 
the  Dq(t )  process  is  strongly  stable  then  the  time  average  expected  ar¬ 
rival  rate  is  less  than  or  equal  to  Q  (ensuring  the  inequality  constraint 
Q  >  q{a)  >  q(a)).  Similarly,  stabilizing  the  other  queues  ensures  the 
other  inequality  constraints  are  satisfied.  This  technique  of  transform¬ 
ing  stochastic  inequality  constraints  into  queueing  stability  problems 
generalizes  the  virtual  power  queue  technique  introduced  in  [109]  for 
stabilizing  wireless  networks  subject  to  average  power  constraints. 

The  goal  of  the  network  controller  is  to  minimize  /(a)  —  g( 7)  while 
stabilizing  all  actual  and  virtual  queues  in  the  network.  To  this  end,  let 
0(f)  =  [U(t),  Dq(t),  Dh(t),  Zx(t),  Zy(t)\  represent  the  combined  net¬ 
work  state  vector,  and  define  the  following  quadratic  Lyapunov  func¬ 
tion: 


L(0) 


MX  My 

+  X]  ’ 

m=  1  1=1 


(6.9) 


where  Di:Q,Dj^  represent  the  i  and  j  entries  of  vectors  Dq  and  D h, 
respectively,  and  Zm<x,  Z[  y  represent  the  m  and  l  entries  of  vectors  Zx 
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and  Zy,  respectively.  Define  the  one-step  Lyapunov  drift  A(0(f))  as 
follows: 

A(©(t))4E  {L(0(t  +  1))  -  L(0(t))  1 ©(*)}• 

Motivated  by  the  Lyapunov  Optimization  Theorem  (Theorem  5.4  of 
Chapter  5),  we  proceed  by  designing  a  controller  that,  every  timeslot, 
minimizes  a  bound  on  the  following  metric: 

Minimize:  A(0(t))  +  DIE  {/(cc(t))  —  5(7^))  |0(t) }  ,  (6.10) 

where  V  >  0  is  a  control  parameter  that  affects  a  performance-delay 
trade-off. 

6.2.1  The  Generalized  CLC  Algorithm  (GCLC) 

An  expression  for  the  drift  metric  (6.10)  can  be  computed  using  the  dy¬ 
namic  queueing  laws  (5.3),  (6.5)-(6.8)  in  the  same  manner  as  in  previous 
chapters.  Below  we  state  the  result,  omitting  the  details  for  brevity: 

A(0(t))  +  CE  {/(a(t))  -  9(j(t))  |0(*)}< 


Jq 

[Qi  l0O)}] 

i=  1 


jh 

-E^>(*)  mhjhMiem-Hj] 

3= 1 
Mx 

-  ^  ZmjX(t)E  {am(t)  -  xm(t)  |©(t)} 

m=  1 
My 

1=1 

+DE  {f{a(t))  -  |0(t) }  ,  (6.11) 

where  B  is  a  finite  and  positive  constant.  Every  timeslot,  the  control 
decision  variables  are  chosen  to  minimize  the  right  hand  side  of  the 
above  inequality. 
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In  order  to  express  the  algorithm  as  a  decoupled  set  of  flow  con¬ 
trol  decisions  and  routing/resource  allocation  decisions,  we  assume  the 
following  separability  criteria : 


•  The  penalty  vector  x{t)  depends  only  on  the  routing  and 
resource  allocation  variables  I(t)  and  /j,^  (t)  (and  not  on  the 
flow  control  variables). 

•  The  reward  vector  y(t)  depends  only  on  the  flow  control  vari¬ 
ables  Rn  ^  (f)  (and  not  on  the  resource  allocation  and  routing 
variables). 


The  above  separability  criteria  leads  to  the  following  dynamic  con¬ 
trol  algorithm. 

Generalized  Cross  Layer  Control  Algorithm  (GCLC): 

Flow  Control:  The  flow  controllers  for  each  active  input  (n,  c)  ob¬ 
serve  the  virtual  and  actual  queue  backlogs,  and  variables  Rn\t)  are 
chosen  to  solve  the  following  optimization  problem: 

My 

Minimize: 

n,c  1=1 

Subject  to:  0  <  R$(t)  <  min[^4^(t)  +  L^\t),  R$], 

The  values  7 i(t)  are  then  computed  as  follows: 

My  Jh 

Minimize:  —  Vg(-y)  +  £  7 lzi,y(t)  7), 

1=1  j= 1 

Subject  to:  0  <  7;  <  Y™ax  for  all  i  G  {1, ... ,  My}. 

The  virtual  queues  Zy(t)  and  Dh(t )  are  then  updated  according  to 
(6.8)  and  (6.6),  using  the  7 (t)  values  computed  above,  and  using  the 
y(t)  rewards  associated  with  the  flow  control  decisions  computed  above. 

Routing /Resource  Allocation:  The  topology  state  S(t)  and  queue 
backlogs  are  observed,  and  the  control  input  /(f)  G  ^S(t)  is  chosen 
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according  to  the  following  optimization: 

Mx 

Minimize:  -  ^  ^  ZmjX(t)xm(t), 

n,b,c  m= 1 

Subject  to:  ^  <  Cnb(I(t),  S(t))  for  all  (n,b), 

C 

1  e  1S(t),  Vnl  (t)  =  o  :  if  (n,  b )  £  Cc, 

where: 

W£\t)±max[u£\t)-Ulc\t),6  • 

Each  link  (n,  6)  then  transmits  /i^  (t)  units  of  commodity  c  data  (using 
idle  fill  if  necessary),  provided  that  wj^(t)  >  0  and  (n,b)  £  Cc. 

The  ot(t)  values  are  then  computed  as  solutions  of  the  following 
optimization  problem: 

Mx  Jq 

Minimize:  V f (a)  —  £  ^2Diiq(t)qi(a), 

m=  1  i=l 

Subject  to:  0  <  am  <  X™ax  for  all  m  £  {1, . . . ,  Mx}. 

The  virtual  queues  Dq(t)  and  Zx(t )  are  then  updated  according  to 
(6.5)  and  (6.7). 

Assuming  the  flow  control  rewards  y\  ( t )  are  associated  with  distinct 
nodes,  and  that  the  utility  functions  g(y)  and  h(y )  are  sums  of  indi¬ 
vidual  utilities  for  each  node,  it  is  not  difficult  to  show  that  the  flow 
control  optimization  can  be  distributed  node  by  node,  using  only  the 
local  (virtual  and  actual)  queue  backlog  information  for  that  node.  The 
routing  algorithm  can  be  further  decoupled  from  resource  allocation  if 
penalties  depend  only  on  1(f)  (and  not  on  the  variables  (t)  if  I(t)  is 
given) ,  in  which  case  an  optimal  commodity  c*  b  can  be  found  as  in  pre¬ 
vious  chapters,  and  the  resulting  resource  allocation  problem  reduces 
to  the  backpressure  algorithm  of  maximizing: 

Mx 

Y,w:b(t)cnb(i(t),s(t))  -  ]T  zmjX(t)xm(t). 

nb  m=  1 


(6.12) 
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The  algorithm  for  choosing  a(t)  can  be  distributed  node-by-node  pro¬ 
vided  that  cost  functions  are  separable.  The  resource  allocation  algo¬ 
rithm  for  choosing  I(t)  to  maximize  (6.12)  is  the  most  complex  part  of 
the  algorithm,  but  is  easily  distributed  in  the  case  when  transmission 
penalty  functions  are  separable  and  each  node  transmits  over  orthog¬ 
onal  frequency  bands,  or  is  approximated  via  methods  discussed  in 
previous  chapters. 


6.2.2  Algorithm  Performance 

Let  Assumptions  1-4  from  Section  6.1.2  hold.  It  is  important  to  note 
that  Assumption  1  is  valid  if  we  assume  either  of  the  following  two 
cases: 


(1)  The  flow  control  rewards  y(t )  are  linear  functions  of  the  de¬ 
cision  variables  R{t)  (the  utilities  h(-)  and  g{  )  can  still  be 
nonlinear).  The  reservoir  buffer  size  is  arbitrary  (infinite,  fi¬ 
nite,  or  zero). 

(2)  The  flow  control  rewards  y(t )  are  arbitrary  functions  of  the 
decision  variables  R(t).  In  this  case,  the  reservoir  buffer  size 
possibly  affects  optimality,  but  Assumption  1  holds  if  we  as¬ 
sume  zero  reservoir  space  (so  that  all  admission/rejection 
decisions  are  made  immediately  upon  arrival)  or  assume  in¬ 
finite  and  always  fidl  reservoir  space. 


Theorem  6.2.  (Algorithm  Performance)  Under  Assumptions  1-4,  the 
GCLC  algorithm  stabilizes  all  actual  and  virtual  queues  in  the  network. 
Furthermore,  there  exists  a  finite  constant  B  (which  can  be  explicitly 
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found  by  computing  the  Lyapunov  drift  of  the  system),  such  that: 
t- 1 


lim  sup  — 


< 


B  +  Vf„ 


t= 0  n,c 


lim  sup  q(x(t))  <  Q, 

t— ►  OO 

lim  inf  h(y(t))  >  H, 

t — >oo 

lim  sup  lf(x(t))  -  g(y(t))]  <  f*  -g*  +  B/V, 

t — >oo 

where  xm(t)=\ E*=o  E  {xm{r)}  and  yt{t)=j  Et=o  E  (^(T)}- 


The  proof  of  the  above  theorem  uses  techniques  similar  to  those 
presented  in  the  previous  chapter  and  developed  in  [109]  [119]  [108], 
and  is  omitted  for  brevity.  It  is  important  to  note  that  the  proof  works 
for  any  system  that  satisfies  Assumptions  1-4  (for  a  given,  potentially 
sub-optimal  constant  c*  =  f*  —  g*  on  the  right  hand  side  of  (6.4)), 
provided  that  the  virtual  queues  are  maintained  as  in  (6.5)-(6.8)  and 
that  control  decision  variables  are  chosen  every  timeslot  to  minimize 
the  right  hand  side  of  (6.11). 

6.3  Energy  Optimal  Networking  Examples 

Here  we  apply  the  general  framework  to  several  important  problems  in 
the  area  of  energy-aware  wireless  networking. 

6.3.1  Max  Throughput  with  Average  Power  Constraints 

Consider  a  multi-user  wireless  downlink  that  transmits  to  L  users 
over  L  distinct  channels.  Let  S(t)  describe  the  collective  channel  state 
process,  and  let  C(P(t),S(t))  =  (Ci(P(t),  S(t)), . .  .  ,CL(P(t),  S(t))) 
represent  the  link  transmission  rate  function,  where  P(t)  = 
(Pi(t), . . .  ,Pi,(t))  is  the  vector  of  power  allocations  for  each  link.  Power 
is  constrained  so  that  E^=i  Bj{t)  <  Pmax  for  all  t.  The  goal  is  to  max¬ 
imize  throughput  subject  to  an  average  power  constraint  Pav  (where 
Pav  <  Pmax)-  Arrival  processes  Ai(t)  satisfy  At(i)  <  R  for  all  i.  Assume 
there  are  no  transport  layer  storage  reservoirs,  so  that  all  arriving  data 
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is  either  admitted  or  dropped.  Let  Ri(t)  represent  the  flow  control  de¬ 
cision  for  queue  i  at  time  t,  and  let  Ui(t)  represent  the  current  backlog 
in  queue  i.  In  this  case,  we  have: 

•  x(t)  =  (Pi{t),...,PL{t)),  y(t)  =  (R1(t),...fRL(t)) 

•  q(P)  =  Eti  pu  Q  =  Pav;  h(R )  =  h  =  o 

•  f(P)  =  o,  g{R)  =  Y.hRi 

Because  h(R)  =  0,  there  is  no  utility  constraint  and  hence  we  do 
not  use  any  Dh(t )  queues.  A  further  simplification  results  by  noticing 
that  the  cost  functions  q(P),  f(P )  are  linear.  Hence,  we  can  avoid  the 
virtual  queues  Zx(t)  and  simply  use  a(t)  =  x(t)  =  P(t)  everywhere 
(including  the  virtual  queue  definitions,  and  in  the  right  hand  side  of 
the  drift  bound  (6.11)),  without  loss  of  optimality.  Likewise,  the  utility 
function  g(R)  is  linear,  and  hence  we  can  avoid  the  virtual  queues  Zy(t ) 
and  simply  use  7(1)  =  y(t)  =  R(t). 

For  the  average  power  constraint,  we  define  a  virtual  power  queue 
Dq(t )  (we  shall  use  D(t)  for  simplicity  here)  with  a  queueing  law  (6.5) 
that  reduces  to  the  following  by  setting  a(t)  =  P(t): 

L 

D(t  +  1)  =  ma x[D(t)  -  Pav,  0]  +  ^  Pi(t).  (6.13) 

i= 1 

with  initial  condition  H(0)  =  0.  In  this  context,  D(t )  has  the  intuitive 
interpretation  of  being  the  accumulated  excess  power  expenditure  over 
and  above  the  average  power  constraint.  With  these  simplifications, 
minimizing  the  right  hand  side  of  (6.11)  leads  to  the  following  Energy 
Constrained  Control  Algorithm  (ECCA)  from  [109]: 

Energy  Constrained  Control  Algorithm  (ECCA)  [109]: 

Flow  Control:  Minimize  ^2f=i[Ui(t)  —  V]Ri(t)  such  that  0  <  Ri(t)  < 
Ai(t )  for  all  i.  That  is,  every  timeslot  and  for  each  queue  i.  we  allow 
the  full  set  of  new  arrivals  Ai(t)  into  the  queue  whenever  Ui(t)  <  V. 
Else,  we  drop  all  new  arrivals  for  queue  i  entering  on  that  timeslot. 

Power  Allocation:  Every  timeslot,  observe  the  current  topology 
state  S(t)  and  the  current  queue  backlogs  U(t)  and  D(t),  and  allocate 
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power  P(t )  =  . . . ,  PL,{t))  according  to  the  following  optimiza¬ 

tion: 

L 

Maximize:  [Ui(t)Ci(P,  S(t))  —  D(t)Pi(t)]  (6.14) 

2—1 
L 

Subject  to:  <  Pmax- 

2—1 

The  virtual  power  queue  D(t)  is  then  updated  via  (6.13). 

From  the  nature  of  the  flow  control  algorithm,  we  have  that  Uft)  < 
V  +  R  for  all  i  and  all  t,  and  hence  the  above  algorithm  trivially  stabi¬ 
lizes  all  actual  queues.  Using  this  fact,  it  is  not  difficult  to  show  that  the 
power  allocation  algorithm  (6.14)  allocates  zero  power  on  any  timeslot 
in  which  the  virtual  queue  D(t )  is  sufficiently  large.  Specifically,  assume 
there  exists  a  constant  (3  >  0  such  that  for  any  link  i,  any  topology  state 
S(t),  and  any  power  vector  P  such  that  Pj  <  Pmax ,  we  have: 

C'i(P,5(2))<C'i(P[z],5(2))+/3Pi, 

where  P  =  (Pi, . . . ,  Pp),  and  P[z]  is  equal  to  P  with  the  exception 
that  the  zth  entry  is  set  to  zero.  It  follows  that  if  D(t)  >  Ui(t)(3,  then 
Pi(t)  =  0.  This  leads  to  the  following  theorem. 

Theorem  6.3.  (ECCA  Performance  [109])  For  any  topology  state 
process  S(t)  and  any  input  process  Aft )  that  satisfies  Aft)  <  R  for  all 
t,  the  ECCA  algorithm  ensures: 

Uft)  <  Umax=V  +  R  for  all  i  and  all  t, 

D(t )  <  Dmax=/3V  +  f3R  +  Pmax  for  all  t. 

Consequently,  the  energy  expended  over  any  interval  of  T  slots  is  never 
more  than  PavT  +  Dmax  (and  so  the  average  power  constraint  clearly 
holds).  Further,  if  the  arrival  vector  A(t )  and  topology  state  S(t)  is 
i.i.d.  over  timeslots,  then  achieved  throughput  satisfies: 

1  t_1  L 

lim  inf  -  EE  ®{Ri(T)}>^r*  -B/V, 

t= 0 i= 1  i 
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where  JA  ri  is  the  optimal  throughput  over  all  possible  policies,  and 
B  is  a  constant  that  can  be  computed  from  the  Lyapunov  drift  of  the 
system  [109]. 


The  maximum  throughput  objective  can  be  replaced  by  a  con¬ 
cave  fairness  objective,  such  as  g(R )  =  +  -Ri)>  in  which 

case  the  flow  control  portion  of  the  ECCA  algorithm  can  be  modi¬ 
fied  by  using  auxiliary  variables  (71(f),  . . . ,  7z,(f))  and  flow  state  queues 
Zy(t)  =  (Zi<y(t), . . . ,  Zi,ty(t))  according  to  the  GCLC  algorithm. 

6.3.2  Minimizing  Power  Expenditure  in  Mobile  Networks 

Consider  a  mobile  wireless  network  with  topology  state  S(t)  and  link 
transmission  rate  function  C(P(t),  S(t))  =  (Cab(P(t),  5(f))),  where 
P(t)  =  (Paft(t))  is  the  matrix  of  power  allocations  over  each  link.  Let  V 
represent  the  set  of  feasible  power  allocation  vectors,  so  that  P(f)  £  V 
for  all  f.  Suppose  that,  without  average  power  constraints  on  each  node, 
the  network  capacity  region  is  given  by  the  set  A. 

Suppose  that  there  is  no  flow  control,  so  that  all  arriving  data  is 
admitted  immediately  into  the  network  layer  (so  that  A$  (f )  =  R $  (f ) 
for  all  f).  Assume  that  the  resulting  traffic  matrix  (A^)  is  in  the  relative 
interior  of  A,  and  define  emax  to  be  the  largest  scalar  such  that  (A^  + 
tmaxAn  ^)  £  A  (where  1^  is  an  indicator  function  that  takes  the  value 
“1”  when  (n,  c)  £  V,  and  “0”  else).  The  goal  is  to  design  a  joint  strategy 
for  resource  allocation,  scheduling,  and  routing,  so  that  the  network  is 
stable  and  total  average  power  expenditure  is  minimized. 

This  objective  can  be  stated  within  our  general  framework  by  defin¬ 
ing  penalties  to  be  the  power  expended  on  each  link:  xab(t )  =  Pab(t)- 
There  are  no  rewards,  and  there  is  only  a  single  cost  function  f(P)  = 
Thab  Pab-  Because  this  function  is  linear  and  there  are  no  additional 
cost  constraints,  we  do  not  require  any  virtual  queues  for  this  system. 
Setting  the  virtual  queue  backlogs  to  zero,  assigning  y(t)  =  7 [t]  =  0, 
and  letting  a(t)  =  P(t)  in  (6.11)  leads  to  the  following  Energy  Efficient 
Control  Algorithm  (EECA)  from  [109] : 

Multi-hop  EECA  for  Minimizing  Average  Power  [109]:  Every 
timeslot,  the  current  U(t)  backlog  and  the  current  topology  state  S(t) 


6.3.  Energy  Optimal  Networking  Examples  117 


is  observed.  Then: 

(1)  For  all  links  (a,  b),  find  the  commodity  c*ab{t)  such  that: 

c*ab(t)  ~  arScg™a\}  {Ua\t)  -  t/fc(c)(t)}  , 

and  define: 

W*ab{t)  ±  ma x[U^\t)  -  Uf“b\t),  0], 

(2)  Power  Allocation:  Choose  P(t)  6  V  to  maximize: 

E  mWCabiP,  S(t))  ~  VPab(t)}  . 

ab 

(3)  Routing:  Over  each  link  (a,  b )  such  that  W*b  >  0,  transmit 
Cab(P(t),  S(t ))  units  of  commodity  c*ab{t)  data  (using  idle  fill 
if  necessary). 

The  above  algorithm  stabilizes  the  network  whenever  emax  >  0,  and 

yields  a  total  average  congestion  of  Yln  c  <  (B  +  V  Pma  x)  /  ^max  and 
ensures  total  average  power  expenditure  is  within  B /V  of  the  minimum 
power  required  for  stability.  If  power  reception  costs  are  also  significant, 
the  algorithm  can  easily  be  modified  by  augmenting  the  penalty  func¬ 
tions  to  account  for  power  costs  expended  by  each  receiver. 

6. 3. 2.1  Simulation  of  a  2-Queue  Downlink  under  EECA 

Consider  the  special  case  of  a  2-queue  downlink  with  time  varying  chan¬ 
nels  and  “Good,”  “Medium,”  and  “Bad”  states  on  each  of  the  two  links. 
At  most  one  link  can  be  activated  for  transmission  every  slot,  and  ex¬ 
actly  1  unit  of  power  is  used  on  each  activation.  Every  timeslot  the 
network  controller  must  choose  to  activate  either  link  1,  link  2,  or  to 
remain  idle  (in  order  to  save  power) .  A  single  packet  can  be  transmitted 
when  a  link  is  in  the  “Bad”  state,  two  packets  can  be  transmitted  in 
the  “Medium”  state,  and  three  can  be  transmitted  in  the  “Good”  state. 
An  example  set  of  arrivals  over  the  course  of  9  timeslots  is  given  in  Fig. 
6.1,  where  the  choices  associated  with  the  MWM  policy  described  in 
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Fig.  6.1  An  example  set  of  arrivals,  channel  conditions,  and  queue  backlogs  for  a  two  queue 
wireless  downlink  under  two  different  scheduling  algorithms,  illustrating  the  power  efficiency 
gains  enabled  by  having  full  knowledge  of  future  arrivals  and  channel  states. 


Section  4.1  are  shown,  along  with  a  more  energy  efficient  set  of  choices, 
both  of  which  leave  the  system  empty  on  timeslot  t  =  9. 

The  EECA  algorithm  in  this  case  reduces  to  serving  the  queue  with 
the  largest  value  of  Ui(t)ni(t )  —  VP  (where  P  =  1  unit)  whenever  this 
value  is  positive,  and  remaining  idle  otherwise.  In  our  simulation,  pack¬ 
ets  arrived  according  to  Poisson  processes  with  rates  Ai  =  8/9,  A2  = 
5/9,  which  are  the  same  as  the  empirical  rates  obtained  by  averaging 
over  the  first  9  timeslots  of  the  example  in  Fig.  6.1.  Channel  states  arise 
as  i.i.d.  vectors  .S/ft))  every  slot.  The  probability  of  each  vector 

state  is  matched  to  the  empirical  occurrence  frequency  in  the  example, 
so  that  Pr[(G,  M)\  =  3/9 ,Pr[(M,B)\  =  2/9,  Pr[(M,  M)]  =  1/9,  etc. 

We  simulated  the  EECA  algorithm  for  20  different  values  of  the 
control  parameter  V,  ranging  from  0  to  104.  Each  simulation  was  run 
for  10  million  timeslots.  In  Fig.  6.2  the  resulting  average  power  is  plot¬ 
ted  against  the  time  average  backlog.  The  corresponding  upper  bound 
derived  in  [109]  is  also  shown  in  the  figure.  We  find  that  average  power 
is  equal  to  0.898  Watts  and  average  sum  backlog  is  2.50  packets  when 
V  =  0  (corresponding  to  the  original  MWM  policy).  Average  power 
decreases  to  its  minimum  value  of  0.518  Watts  as  the  control  parame¬ 
ter  V  is  increased,  with  a  corresponding  tradeoff  in  average  delay.  As 
a  point  of  reference,  we  note  that  at  V  =  50,  the  average  power  is  0.53 
Watts  and  the  average  sum  backlog  in  the  system  is  21.0  packets. 
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Average  Power  versus  Average  Backlog 


Fig.  6.2  Average  power  versus  average  backlog  for  a  two  queue  downlink  under  the  EECA 
algorithm  (from  [109]). 


6. 3. 2. 2  Minimum  Energy  Scheduling  for  Mobile  Networks 

Here  we  consider  an  ad-hoc  mobile  network  with  28  users  and  a  cell 
structure  arranged  as  a  4  x  4  grid,  as  shown  in  Fig.  6.3.  For  simplicity, 
we  assume  there  can  be  at  most  one  transmission  per  cell  per  times¬ 
lot,  and  that  all  transmissions  use  full  power  of  1  Watt.  We  assume 
transmission  rates  are  adaptive,  and  that  3  packets  can  be  transferred 
if  the  receiver  is  in  the  same  cell  as  the  transmitter,  while  only  1  packet 
can  be  transferred  if  the  receiver  is  in  one  of  the  adjacent  cells  to  the 
North,  South,  East,  or  West.  Data  arrives  to  each  node  according  to  a 
Bernoulli  arrival  process  with  rate  A  =  0.5  packets/slot  (so  that  a  single 
packet  arrives  with  probability  0.5,  else  no  packet  arrives).  We  assume 
source-destination  pairs  are  given  by  the  grouping  1  2,  3  4,  . . . , 

27  28,  so  that  node  1  packets  are  destined  for  node  2  and  node  2 

packets  are  destined  for  node  1,  node  3  packets  are  destined  for  node 
4  and  node  4  packets  are  destined  for  node  3,  etc. 

We  simulate  the  multi-hop  EECA  algorithm  for  both  a  Markovian 
random  walk  model  and  an  i.i.d.  mobility  model,  with  the  objective  of 
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Energy  vets  us  Delay  for  an  ad-hcc  mobile  network 


Fig.  6.3  An  ad-hoc  mobile  network  with  adaptive  transmission  rates,  and  the  resulting  per 
node  average  power  expenditure  versus  average  node  congestion  for  V  between  0  and  200. 


minimizing  total  power  expenditure.  In  the  Markovian  mobility  model, 
every  timeslot  nodes  independently  move  to  a  neighboring  cell  either 
to  the  North,  South,  East,  or  West,  with  equal  probability.  In  the  case 
when  a  node  on  the  edge  of  the  network  attempts  to  move  in  an  infea¬ 
sible  direction,  it  simply  stays  in  its  current  cell.  In  the  i.i.d.  mobility 
model,  nodes  randomly  choose  new  cell  locations  every  timeslot  inde¬ 
pendently  and  uniformly  over  the  set  of  all  16  cells.  It  is  not  difficult 
to  show  that  both  mobility  models  have  the  same  steady  state  node 
location  distribution.  Hence,  the  network  capacity  region  and  the  min¬ 
imum  average  power  required  for  stability  are  exactly  the  same  for  both 
mobility  models  [109]  [108].  In  this  case,  the  minimum  average  power 
for  stability  under  the  given  traffic  load  can  be  exactly  computed,  and 
is  equal  to  0.303  Watts  [160]. 

Simulations  were  conducted  using  control  parameters  V  in  the  range 
from  0  to  200,  and  the  results  are  given  in  Fig.  6.3.  In  the  figure,  each 
data  point  represents  an  independent  simulation  for  a  particular  value 
of  V  over  the  course  of  4  million  timeslots.  The  resulting  per-node  aver¬ 
age  power  is  plotted  against  the  resulting  per-node  average  queue  con- 
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gestion.  From  the  figure,  it  is  clear  that  under  both  mobility  models, 
average  power  expenditure  quickly  converges  to  the  minimum  power 
level  as  the  control  parameter  V  is  increased  (and  hence,  delay  is  in¬ 
creased).  The  average  delay  under  Markovian  mobility  is  slightly  larger 
than  the  delay  under  i.i.d.  mobility.  As  an  example  set  of  data  points, 
we  note  that  for  the  Markovian  mobility  model  at  V  =  0  (correspond¬ 
ing  to  the  Dynamic  Backpressure  strategy  of  Chapter  4),  the  per-node 
average  backlog  is  89.2  packets  (about  3.3  packets  on  average  in  each 
of  the  27  internal  queues),  and  per-node  average  power  expenditure  is 
0.477  Watts.  At  V  =  40,  the  per-node  average  backlog  is  263.6  packets, 
and  per-node  average  power  expenditure  is  0.305  Watts.  For  values  of 
V  beyond  50,  average  power  expenditure  differs  from  the  optimal  value 
of  0.303  only  in  the  fourth  or  fifth  significant  digits,  while  average  con¬ 
gestion  continues  to  increase. 

6.3.3  Notes 

•  Related  work  by  Liu,  Chong,  and  Shroff  in  [96]  considers 
maximizing  a  time  average  utility  function  for  an  infinitely 
backlogged  downlink  in  the  case  when  exactly  one  user  must 
transmit  on  each  and  every  timeslot.  The  technique  can  be 
used  to  address  the  minimum  average  power  problem  if  all 
transmissions  consist  of  single  packets  and  “utility”  is  mea¬ 
sured  by  a  negative  power  cost  associated  with  transmitting 
a  packet  in  the  current  channel  state.  In  this  case,  the  algo¬ 
rithm  of  [96]  chooses  to  transmit  over  the  link  i  that  maxi¬ 
mizes  v*  — Pi ,  where  are  pre-computed  indices  deter¬ 

mined  by  the  system  constraints  and  the  channel  state  prob¬ 
abilities.  The  EEC  A  algorithm  applied  to  this  setting  would 
choose  the  link  i  that  maximizes  the  index  Ui(t)/V  —  Pi,  so 
that  the  pre-computed  index  v*  is  replaced  by  a  time  varying 
index  Ui(t)/V  that  does  not  require  any  pre-computation  or 
prior  knowledge  of  channel  probabilities.  Note  also  that  per¬ 
formance  can  be  improved  if  the  system  is  allowed  to  be  idle 
during  some  timeslots,  in  which  case  EEC  A  would  enter  idle 
mode  whenever  Ui(t)/V  —  Pj  is  non-positive  for  all  links  i. 
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•  The  EECA  and  ECCA  algorithms  presented  in  this  chapter 
generalize  several  related  results.  Work  in  [153]  describes  the 
information  theoretic  capacity  region  for  multiple  access  fad¬ 
ing  channels,  and  a  dynamic  strategy  is  given  there  for  the 
case  when  all  users  are  infinitely  backlogged  and  all  channel 
probabilities  are  a-priori  known.  This  can  be  viewed  as  a  spe¬ 
cial  case  of  our  framework,  where  rate-power  curves  C(P,  S ) 
are  determined  by  an  optimal  receiver  algorithm  that  uses 
successive  noise  cancellation  every  slot.  Broadcast  capacities 
for  power  limited  downlinks  are  developed  in  [89]  [64]  as¬ 
suming  infinite  backlog  and  known  channel  probabilities,  and 
a  capacity  achieving  queueing  strategy  with  average  energy 
constraints  is  developed  in  [168]  under  the  assumption  that 
channel  probabilities  are  known. 

•  Average  power  expenditure  in  an  ad-hoc  network  with  no 
channel  or  topology  state  variation  is  considered  in  [35]  [76] 
[20] .  In  [35] ,  a  periodic  scheduling  framework  is  developed  for 
minimizing  energy  expenditure  subject  to  fixed  time  average 
rate  requirements  on  each  link.  The  solution  is  based  on  an 
off-line  computation  of  an  optimization  problem.  The  work 
in  [76]  considers  complexity  issues  for  a  similar  problem,  and 
the  work  in  [20]  treats  a  network  with  simpler  interference 
constraints  and  provides  an  algorithm  for  computing  sched¬ 
ules  that  are  within  a  constant  factor  of  the  minimum  energy 
solution.  The  EECA  algorithm  provides  an  on-line  solution 
strategy  for  all  of  these  problems,  which  can  also  be  used  as 
an  off-line  computation  method  for  the  scenarios  of  [35]  [76] 
[20]. 

•  The  [0(1/V),  0(V)\  energy-delay  tradeoff  achieved  by  the 
EECA  algorithm  is  not  the  optimal  tradeoff.  Work  by  Berry 
and  Gallager  in  [15]  considers  the  energy-delay  tradeoff  for 
a  single  wireless  link  with  a  random  arrival  process,  a  fading 
channel,  and  a  concave  rate-power  curve.  The  fundamental 
tradeoff  law  is  shown  to  have  a  square-root  structure  with 
energy-delay  parameters  \0(1/V),0(\/V)\.  This  result  is  re¬ 
cently  extended  to  multi-user  networks  in  [114].  An  improved 
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tradeoff  [0( l/V),0{y/V)\  is  achievable  in  the  exceptional 
cases  when  the  system  has  a  piecewise  linear  structure  [114] 
or  when  a  small  fraction  of  packets  can  be  dropped  [112]. 
Related  notions  of  delay  limited  capacities  are  developed  in 
[154]  for  the  case  when  tolerable  delay  is  no  more  than  one 
timeslot. 

•  Energy  and  delay  issues  for  static  wireless  links  are  consid¬ 
ered  in  [161]  [170]  [30],  and  a  problem  with  a  static  link  but 
stochastic  arrivals  is  treated  using  filter  theory  in  [73] .  Similar 
problems  of  minimizing  energy  subject  to  delay  constraints 
or  minimizing  delay  subject  to  energy  constraints  are  treated 
for  single  link  satellite  and  wireless  systems  in  [170]  [49]  [50] 
[56]  using  stochastic  differential  equations,  dynamic  program¬ 
ming,  and  Markov  decision  theory,  and  multi-link  systems  are 
considered  in  [141].  Related  problems  for  wireless  sensor  net¬ 
works  are  considered  in  [37]  [169],  and  delay  efficient  sleep 
scheduling  is  considered  in  [98].  Asymptotic  energy  results 
as  network  size  is  scaled  to  infinity  are  considered  in  [90]. 

6.4  A  Related  Algorithm 

In  this  section  we  describe  another  approach  for  the  solution  to  the 
general  optimization  problem  (6.1),  proposed  by  Stolyar  [136],  under 
modeling  assumptions  similar  to  those  described  above.  We  describe 
next  the  assumptions  in  [136].  For  consistency  we  employ  the  notation 
used  in  the  current  paper. 

(1)  The  network  model  and  the  set  of  penalties  xm,  rn  E  A4  are 
the  same  as  those  described  in  Section  6.1.1. 

(2)  The  network  topology  state  S(t)  constitutes  an  irreducible 
finite  Markov  Chain  with  state  space  S. 

(3)  For  each  state  s  €  S,  the  set  of  available  controls  Ts  is  finite. 

(4)  When  S  (t)  =  s  and  a  control  I  £  Zs  is  chosen,  the  following 
occurs. 

(a)  A  penalty  xm(I)  is  generated  for  all  m  G  M. 

(b)  An  amount  Rn(I )  of  exogenous  traffic  is  admitted  to 
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enter  node  i  £  M . 

(c)  Each  node  i  £  M  selects  up  to  Hi{I)  units  of  traffic 
from  its  queue  (if  there  are  fewer  units  than  //,;(/), 
the  whole  queue  is  emptied).  Each  of  these  units  is 
routed  to  the  rest  of  the  nodes  in  the  network  with 
probabilities  Pij(I),  j  £  M . 

(d)  When  a  traffic  unit  reaches  its  destination,  it  is  re¬ 
moved  from  the  network. 

The  penalties  xm(I),  the  admitted  traffic  Rn(I),  and  the  traffic 
transmitted  by  each  node  fii(I)  may  also  be  random  variables  that 
depend  on  the  control  chosen,  but  are  independent  of  anything  else. 
The  routing  probabilities  pik  ( I )  may  depend  on  the  control  but  are  also 
independent  of  anything  else.  Multicommodity  flows  can  be  taken  care 
of  by  considering  multiple  queues  at  each  node,  one  for  each  commodity 
along  the  lines  described  in  earlier  sections. 

The  Markovian  assumption  on  the  S(t)  topology  state  is  similar 
in  both  our  general  framework  and  the  framework  of  [136].  The  as¬ 
sumption  that  the  available  control  sets  Ts  (s  £  S)  are  finite  does  not 
include  the  case  where  controls  may  take  continuous  values,  as  for  ex¬ 
ample,  when  the  control  refers  to  transmission  power  of  a  node  which 
may  be  continuously  varied.  The  assumption  that  Rn(I)  depends  only 
on  the  control  chosen  excludes  the  case  of  using  a  reservoir  for  holding 
traffic  that  has  not  been  admitted  to  the  network.  These  assumptions 
are  not  very  restrictive  and  they  can  probably  be  relaxed  without  af¬ 
fecting  the  optimality  of  the  proposed  algorithm  to  be  described  below. 
The  assumption  imposed  on  routing,  i.e. ,  the  use  of  routing  probabil¬ 
ities,  may  be  more  restrictive  as  it  does  not  seem  easy  to  include  in  a 
simple  manner  the  practical  case  where  a  deterministic  part  of  the  traf¬ 
fic  m{I)  selected  by  node  i  is  routed  to  node  j,  rather  than  a  random 
amount  with  average  pij  (I)  =  pi(I)pij(I).  However  in  several  specific 
problems,  if  one  formally  uses  the  average  amount  as  the  amounts  ac¬ 
tually  routed,  the  algorithm  seems  to  work.  Therefore,  it  seems  that 
this  assumption  is  mainly  needed  for  the  proofs  in  [136]  and  may  also 
be  relaxed  without  affecting  the  optimality  of  the  algorithm.  Further, 
in  the  special  case  when  each  node  is  restricted  to  sending  over  a  single 
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outgoing  link,  the  probabilistic  routing  can  be  designed  to  choose  the 
desired  link  with  probability  1. 

The  system  is  slotted.  The  proposed  algorithm  performs  the  follow¬ 
ing  at  the  beginning  of  each  slot. 

(1)  The  weighted  averages  of  penalties  are  measured, 

xm{t)  =  (1  -  P)xm(t  -  1)  +  / 3xm(I(t  -  1)),  m  £  M,  (6.15) 

where  0  <  /3  <  1  is  a  typically  small  number.  The  smaller  /3  is,  the 
closer  the  performance  of  the  algorithm  approaches  the  optimum,  at 
the  expense  of  less  adaptivity  of  the  algorithm  in  case  of  parameter 
changes. 

(2)  The  queue  sizes,  Un(t),  n  £  M  are  measured  and  the  network  state 
S  (t)  is  observed.  The  control  to  be  employed  is  obtained  as  the  solution 
of  the  following  optimization  problem. 

Max:  (3  Y.  ( Ui(t )  -  Uj  (t))  m,  (/)  -  p  Y, Ui  (*)  Rn  (J)  (6-16) 

i,j  i 

~  ^2  W  (Xm{t))  /tixm(t))xm(I), 
m&M 

subject  to:  I  £  ^s(t)  ■ 

Here  we  are  assuming  that  the  algorithm  transmits  a  deterministic 
amount  of  bits  Rij(I)  over  each  link  rather  than  the  probabilistic 

amount  formally  used  in  the  framework  of  [136]. 

There  are  similarities  as  well  as  differences  between  this  algorithm 
and  the  algorithm  presented  in  Section  6.2.1.  These  are  best  explained 
by  an  example  which  we  discuss  below. 


Example  6.1.  Consider  a  wireless  network  with  L  transmission  links 
denoted  by  a  link  set  C.  The  transmission  rate  for  each  link  l  depends 
on  a  power  allocation  vector  according  to  the  function  Ci(P(t),  S(t)). 
Suppose  that  power  must  satisfy  P(t)  £  V  for  all  t,  where  V  is  a  finite 
collection  of  acceptable  power  allocation  vectors.  We  consider  single¬ 
hop  transmissions  where  exogenous  arrivals  Ai(t)  arrive  to  the  network 
at  the  source  node  of  link  l  and  must  be  transmitted  over  link  l.  The 
exogenous  arrival  vectors  (A/  (t))/g£  are  i.i.d.  over  timeslots,  and  we 
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assume  that  A[(t)  <  Amax.  At  each  timeslot,  Ri  (t)  bits  from  this  ex¬ 
ogenous  traffic  may  enter  the  network,  where  Ri(t)  <  Ai  (t).  Traffic  that 
does  not  enter  the  network  is  dropped,  i.e. ,  we  assume  zero  reservoir 
space.  The  arrival  rate  vector  may  take  arbitrary  values,  and  in  partic¬ 
ular  may  lie  outside  the  capacity  region  of  the  system.  The  goal  is  to 
design  a  joint  flow  control  and  resource  allocation  algorithm  that  stabi¬ 
lizes  the  network  and  yields  an  optimal  throughput  utility  Yhecdlfrl), 
where  gi(r)  denotes  a  concave  utility  function  of  the  throughput  over 
link  l,  and  fi  represents  the  long  term  average  admitted  rate  into  link  l 
(assuming  for  now  such  time  averages  exist).  In  the  following  we  will  re¬ 
fer  to  the  GCLC  algorithm  presented  in  Section  6.2.1  as  ALG1,  and  to 
the  Algorithm  presented  in  [136]  as  ALG2.  According  to  the  description 
in  Section  6.2.1,  ALG1  consist  of  the  following  operations. 

Description  of  ALG1:  In  this  example,  the  GCLC  algorithm  uses 
rewards  y(t)  =  R(t)  (where  0  <  Ri(t)  <  Amax),  with  utility  function 
g(R)  =  Yliec9l(Rl)-  There  are  no  utility  constraints,  cost  constraints, 
or  penalties,  and  so  queues  Dh(t),  Dq(t),  Zx(t )  are  not  required.  As  the 
utility  function  g(R)  is  non-linear  and  concave,  the  auxiliary  variables 
and  flow  state  queues  7 (t)  and  Zy(t)  are  used.  For  notational  simplicity, 
we  drop  the  subscript  on  queue  Zy(t)  and  use  instead  Z{t).  The  initial 
condition  is  Z( 0)  =  0,  and  the  queue  update  equation  (6.8)  in  this 
example  is  given  by: 

Z(t  +  1)  =  ma x[Z(t)  —  R(t),  0]  +  7 (f)  (6.17) 

This  algorithm  thus  reduces  to  the  CLC2b  algorithm  of  Chapter  5. 
In  particular,  every  slot  the  flow  control  variables  Ri(t)  are  chosen 
such  that  Ri(t)  =  A[(t)  whenever  Ui(t)  <  Z/(t),  and  Ri(t)  =  0  else. 
The  values  7 i(t)  are  then  computed  as  the  solution  to  the  following 
optimization: 

Minimize:  -  V 9/(7*)  +  7 iZi(t)  (6.18) 

Subject  to:0  <7 1<  Amax. 

The  virtual  queue  Z(t)  is  then  updated  according  to  (6.17).  The  re¬ 
source  allocation  strategy  is  then  the  same  as  the  Dynamic  Backpres¬ 
sure  strategy  of  Section  4.3,  where  P(t)  €  V  is  chosen  to  maximize 
t/,(i)C)(P(f),S(f)). 
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Description  of  ALG2:  To  bring  the  problem  into  a  cost- 
minimization  form  we  define  penalty  variables  xi  ( t )  =  Hmax  —  Ri  (t) 
and  consider  the  equivalent  problem  of  minimizing 

(6-19) 

leC 

where 

ft  (X)  =  m(A™x)  -  9l(A™x  -  x).  (6.20) 

Hence  the  “penalties”  are  now  (xi(t))leC  ■  Since  xi  (t)  <  Amax  we  may 
set  XjAax  =  Hmax,  l  G  C.  With  the  zero-reservoir  assumption,  and  the 
condition  that  Ai  (t)  takes  finite  values,  we  may  consider  that  the  vector 
( Ai(t))leC  is  part  of  network  state  S  (t)  and  that  Rift)  is  part  of  the 
exercised  control  1g(t)  (interpreted  as  a  control  vector  augmented  with 
the  flow  control  choice  and  with  a  state  that  includes  the  new  arrivals). 
The  ALG2  then  keeps  track  of  the  weighted  averages: 

xi(t  +  1)  =  (1  —  (3)xi (t)  +  /3  (Amax  —  Ri  ( t )) ,  2  £  £,  (6.21) 

Taking  into  account  the  form  of  the  function  (6.19),  the  optimization 
problem  (6.16)  for  determining  the  optimal  control  2 becomes  in 
this  case: 

Max:  (3 

lec  lec 

-E 

lec 

0  <  Ri  <  At(t),  l  €  C  f  P(t)  e  V,  (6.22) 

where  f{  (x)  denotes  the  derivative  of  fi  (x).  Observing  the  structure  of 
this  problem  it  can  be  seen  that  it  is  decomposable  into  the  following 
subproblems. 

Flow  Control:  The  accepted  traffic  on  link  l,  Ri(t),  is  given  by  the 
solution  to  the  following  optimization  problem, 

Minimize:  /3 

lec  lec 

o  <n<  Ai(t),  i  g  c. 
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The  solution  to  this  problem  is  simply  to  accept  all  traffic  Ai  (t)  if 
Ui  (t)  <  fj  ( xi(t ))  / /3  and  reject  it  otherwise.  Using  the  computed  values 
of  (Ri  (t))i gjC,  the  auxiliary  queues  are  updated  according  to  (6.21). 

Resource  Allocation:  Same  as  in  ALG1. 

The  main  difference  between  the  two  algorithms  is  that  ALG1  uses 
the  auxiliary  queue  updates  (6.17),  while  ALG2  uses  the  weighted  av¬ 
erages  updates  (6.21).  It  is  remarkable  that  both  algorithms  introduce 
additional  state  variables  to  solve  the  non-linear  optimization.  The  up¬ 
dates  performed  by  either  algorithm  have  an  effect  on  the  adaptivity  of 
the  system  when  channel  state  or  arrival  statistics  change.  It  would  be 
interesting  to  assess  the  delay  properties  of  both  algorithms,  and  the 
adaptability  capabilities  of  the  two  algorithms  when  statistical  parame¬ 
ters  change.  The  ALG2  requires  utility  functions  to  have  computable 
derivatives  in  order  to  implement  the  flow  control  algorithm.  This  is  not 
a  major  assumption,  as  most  utilities  of  interest  are  simple  differentiable 
functions.  The  ALG1  does  not  require  the  existence  of  a  derivative,  but 
requires  the  solution  of  problem  (6.18)  to  implement  the  flow  control. 
In  this  case,  the  problem  is  a  simple  concave  maximization  over  a  single 
variable,  and  so  the  solution  is  either  one  of  the  two  endpoints,  or  a 
local  maximum  in  the  interior.  In  the  case  when  a  derivative  exists, 
any  such  local  maximum  would  satisfy  g'^i)  =  Zi(t)/V,  which  can  be 
solved  in  closed  form  if  the  inverse  of  the  derivative  is  known.  In  both 
cases,  the  flow  control  algorithms  are  quite  simple.  The  main  computa¬ 
tional  complexity  is  in  the  solution  to  the  Link  Power  Control  problem 
in  an  efficient  and  preferably  distributed  manner. 
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Final  Remarks 


The  cross/layer  resource  allocation  model  presented  in  this  paper  was 
motivated  by  wireless  communication  networks.  Nevertheless  it  has 
within  its  scope  a  number  of  other  application  areas  and  it  can  be 
extended  in  various  ways.  Some  are  outlined  below. 

High  speed  switches  with  input  queueing  is  an  area  that  attracted 
a  lot  of  attention  [104]  [40]  [72]  [88]  [87]  [102],  Packet  scheduling  in 
such  a  switch  falls  within  the  scope  of  our  resource  allocation  model. 
The  switch  architecture  imposes  restrictions  of  a  single  packet  trans¬ 
mission  per  slot  for  each  input  and  output  port,  a  restriction  that  is 
captured  by  the  transmission  conflict  constraints  of  the  resource  alloca¬ 
tion  model.  The  capacity  region  characterization  as  well  as  the  optimal 
control  policies  results  apply  in  that  case.  Different  applications  pose 
different  requirements  on  the  scheduling  policy  regarding  tolerable  com¬ 
plexity  and  distributed  versus  centralized  implementations.  In  the  case 
of  the  switch  for  instance,  the  computational  burden  of  scheduling  is 
severely  constrained  by  the  time  for  packet  transmission  and  the  latter 
is  shrinking  as  the  bandwidth  increases.  Another  approach  to  deal  with 
high  scheduling  complexity  is  to  resort  to  randomized  scheduling  poli¬ 
cies  [145]  [126]  [54] .  In  these  polices  a  randomized  algorithm  computes 
the  access  schedule  at  each  time  and  it  updates  the  one  used  previously 
only  if  it  is  better.  The  randomized  algorithm  being  of  low  computa¬ 
tional  complexity  simplifies  the  computational  requirements,  without 
sacrificing  any  throughput  but  only  with  some  increase  of  the  delay. 
When  the  network  is  geographically  distributed,  collecting  state  infor- 
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mation  for  the  access  controller  might  be  cumbersome  and  might  result 
in  outdated  information  available  to  the  controller.  Recently  there  has 
been  several  efforts  to  identify  scheduling  algorithms  with  low  compu¬ 
tational  complexity  and  amenable  to  distributed  implementation.  The 
impact  of  imperfect  scheduling  on  the  throughput  in  cases  where  sub- 
optimal  algorithms  perform  the  scheduling  is  studied  in  [29]  [46]  [92] 
while  fully  distributed  scheduling  policies  are  given  as  well. 

Resource  allocation  problems  in  manufacturing  and  transportation 
that  have  been  considered  recently  fall  within  the  scope  of  the  model 
we  consider  here  as  well.  An  extension  of  the  back  pressure  policy  that 
is  applicable  in  systems  with  random  service  times  and  non-preemptive 
service  is  presented  in  [146].  The  maximum  pressure  policy  proposed  in 
[38]  [39]  follows  similar  principles  with  the  adaptive  back  pressure  pol¬ 
icy  while  it  was  shown  to  possess  certain  optimality  properties.  Other 
service  provisioning  structures  that  fall  within  the  scope  of  the  pre¬ 
sented  model  have  been  considered  in  [9]  [13]  [14].  The  analysis  of  the 
system  in  the  latter  cases  was  done  under  general  stationary  ergodic 
assumptions  about  the  statistics. 

Various  load  balancing  and  routing  problems  studied  by  the  theo¬ 
retical  computer  science  community  fall  within  the  scope  of  the  model 
we  consider  here,  while  various  policies  proposed  in  that  context  rely 
on  the  differential  backlog  rule  for  traffic  forwarding.  A  policy  similar 
to  adaptive  back  pressure  policy  was  proposed  and  studied  in  [2]  [11] 
[8],  in  the  context  of  adversarial  queueing  theory.  That  is,  its  perfor¬ 
mance  was  analyzed  under  arrival  traffic  patterns  that  might  be  the 
worst  possible  within  a  certain  family  of  arrival  patterns,  for  instance 
all  possible  arrival  patterns  at  the  output  of  a  traffic  regulator  that  pro¬ 
duces  traffic  constrained  bursts.  It  was  shown  that  the  policy  achieves 
maximum  throughput  in  that  context  as  well. 

The  problem  of  paging  a  mobile  user  may  be  casted  in  the  context 
of  our  model  and  the  adaptive  back  pressure  strategy  provides  optimal 
paging  policies  [6]  [7] . 

Multicasting  can  be  viewed  as  a  generalization  of  unicast  informa¬ 
tion  transport,  where  the  information  generated  at  source  node  s;  needs 
to  be  transported  to  all  nodes  of  the  set  of  destinations  D/.  The  capac¬ 
ity  and  maximum  throughput  results  presented  in  the  paper  can  be 
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extended  to  the  case  of  multicasting  as  is  reported  in  [131]  [132],  In¬ 
formation  flows  in  multicasting  are  identified  by  the  source  node  and 
the  group  of  receivers  and  typically  there  are  several  of  them  flowing 
through  the  network  (si,  Hi), (sl,  Dl).  A  multicast  information  flow 
might  be  served  by  an  eligible  multicast  tree,  i.e.  a  directed  tree  rooted 
at  si  and  including  in  its  leaf  or  intermediate  nodes,  all  nodes  of  D[.  The 
information  flows  from  the  source  to  the  destination  nodes  through  the 
unique  paths  designated  by  the  multicast  tree.  In  that  sense  informa¬ 
tion  transport  in  multicasting  resembles  more  virtual  circuit  forwarding 
rather  than  datagram.  The  traffic  of  an  information  flow  might  be  split 
across  all  multicast  trees  eligible  for  the  specific  flow  that  are  desig¬ 
nated  to  carry  traffic  from  that  flow.  A  vector  of  the  traffic  loads  of 
each  information  flow  is  feasible,  if  there  is  a  way  to  split  the  load  of 
each  information  flow  across  the  eligible  trees  for  the  flow  such  that 
the  aggregate  load  of  the  traffic  of  all  trees  results  in  traffic  loads  for 
the  links  that  do  not  exceed  their  capacities.  The  transport  capacity 
region  is  defined  to  include  all  feasible  traffic  load  vectors  as  above.  A 
maximum  throughput  policy  for  the  multicast  case  can  be  obtained  by 
a  variation  of  the  adaptive  back  pressure  policy  as  follows.  Transmis¬ 
sion  priorities  among  the  different  trees  crossing  a  certain  link  are  given 
according  to  the  sizes  of  the  differential  backlogs,  where  the  differential 
backlog  of  a  tree  in  a  certain  link  is  the  difference  of  the  upstream  node 
backlog  minus  the  maximum  of  the  backlogs  at  each  one  of  the  links 
where  the  tree  branches  out  at  the  downstream  node.  The  above  rule, 
in  combination  with  a  load  balancing  rule  for  allocating  the  traffic  of 
an  information  flow  to  the  various  eligible  trees  that  may  support  it 
gives  a  maximum  throughput  transmission  control  policy. 

In  the  above  discussion  about  multicasting  the  information  of  the 
different  flows  after  it  enters  the  network  remains  intact  until  it  reaches 
the  destinations.  An  alternative  that  promises  significant  capacity 
gains,  for  multicasting  in  particular,  is  to  allow  for  combining  of  the 
information  of  different  flows  through  linear  or  nonlinear  operations 
at  intermediate  network  nodes.  That  approach,  referred  to  as  network 
coding,  [1]  is  intensely  investigated  recently.  It  is  possible  to  use  an  ap¬ 
proach  along  the  lines  of  the  adaptive  back  pressure  control  in  order  to 
achieve  maximum  throughput  in  the  case  where  there  is  intra-session 
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coding.  That  approach  is  pursued  in  [62]  where  it  is  shown  how  the 
capacity  region  including  network  coding  can  be  achieved. 
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